You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Bhavesh Shah <bh...@gmail.com> on 2012/01/17 06:24:18 UTC
Problem related to Hive - LOAD DATA
Hello,
I am using hive LOAD DATA command.
I am writing some content in one text file and loading that content into
table every time when I write that file.
But when I use LOAD DATA command:
"load data local inpath '/home/hadoop/quarter.txt' into table quartertable"
In HDFS, for the first time I saw file quarter.txt after loading,
but later when I tried for multiple times,
in HDFS, I saw loaded files as:
quarter.txt
quarter.txt_copy_1
quarter.txt_copy_2
quarter.txt_copy_3
quarter.txt_copy_4
And when I access this quartertable I get the data from all this files.
What should I do to maintain just one file and to keep overwrite it?
Pls suggest me.
Thanks.
--
Regards,
Bhavesh Shah
Re: Problem related to Hive - LOAD DATA
Posted by Bhavesh Shah <bh...@gmail.com>.
Hi Bejoy.K.S,
Thanks for your help.
--
Regards,
Bhavesh Shah
On Tue, Jan 17, 2012 at 11:07 AM, Bejoy Ks <be...@yahoo.com> wrote:
> Hi Bhavesh
> This is the expected behavior from hive, If you want to overwrite
> the data in table on new data loads use OVERWRITE. LOAD DATA without
> OVERWRITE is used to just append new data to existing data in a hive table.
> load data local inpath '/home/hadoop/quarter.txt' OVERWRITE into table
> quartertable
>
> Then about
> quarter.txt_copy_1
> quarter.txt_copy_2
> quarter.txt_copy_3
> quarter.txt_copy_4
>
> Since the target hdfs dir already has a quarter.txt, the next files are
> renamed during hdfs copy.
>
> Hope it helps!...
>
>
> Regards
> Bejoy.K.S
>
> ------------------------------
> *From:* Bhavesh Shah <bh...@gmail.com>
> *To:* user@hive.apache.org
> *Sent:* Tuesday, January 17, 2012 10:54 AM
> *Subject:* Problem related to Hive - LOAD DATA
>
> Hello,
> I am using hive LOAD DATA command.
> I am writing some content in one text file and loading that content into
> table every time when I write that file.
>
> But when I use LOAD DATA command:
> "load data local inpath '/home/hadoop/quarter.txt' into table quartertable"
>
> In HDFS, for the first time I saw file quarter.txt after loading,
> but later when I tried for multiple times,
> in HDFS, I saw loaded files as:
> quarter.txt
> quarter.txt_copy_1
> quarter.txt_copy_2
> quarter.txt_copy_3
> quarter.txt_copy_4
>
> And when I access this quartertable I get the data from all this files.
>
> What should I do to maintain just one file and to keep overwrite it?
> Pls suggest me.
> Thanks.
>
>
>
> --
> Regards,
> Bhavesh Shah
>
>
>
>
Re: Problem related to Hive - LOAD DATA
Posted by Bejoy Ks <be...@yahoo.com>.
Hi Bhavesh
This is the expected behavior from hive, If you want to overwrite the data in table on new data loads use OVERWRITE. LOAD DATA without OVERWRITE is used to just append new data to existing data in a hive table.
load data local inpath '/home/hadoop/quarter.txt' OVERWRITE into table quartertable
Then about
quarter.txt_copy_1
quarter.txt_copy_2
quarter.txt_copy_3
quarter.txt_copy_4
Since the target hdfs dir already has a quarter.txt, the next files are renamed during hdfs copy.
Hope it helps!...
Regards
Bejoy.K.S
________________________________
From: Bhavesh Shah <bh...@gmail.com>
To: user@hive.apache.org
Sent: Tuesday, January 17, 2012 10:54 AM
Subject: Problem related to Hive - LOAD DATA
Hello,
I am using hive LOAD DATA command.
I am writing some content in one text file and loading that content into table every time when I write that file.
But when I use LOAD DATA command:
"load data local inpath '/home/hadoop/quarter.txt' into table quartertable"
In HDFS, for the first time I saw file quarter.txt after loading,
but later when I tried for multiple times,
in HDFS, I saw loaded files as:
quarter.txt
quarter.txt_copy_1
quarter.txt_copy_2
quarter.txt_copy_3
quarter.txt_copy_4
And when I access this quartertable I get the data from all this files.
What should I do to maintain just one file and to keep overwrite it?
Pls suggest me.
Thanks.
--
Regards,
Bhavesh Shah