You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by "Connell, Chuck" <Ch...@nuance.com> on 2012/07/25 17:21:55 UTC
Problem replacing existing Hive file with modified copy
I created a Hive table that consists of two files, names1.txt and names2.txt. The table works correctly and answers all queries etc.
I want to REPLACE names2.txt with a modified version. I copied the new version of names2.txt to the /tmp/input folder within HDFS. Then I tried the command:
hive -e "load data inpath '/tmp/input/names2.txt' into table names"
I get an error: "Failed with exception null. Exception error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask"
Is this just a bug in this version (hadoop-hive-0.7.1+42.43-2.noarch, via CDH3 distro)?
Is my syntax correct to update a file in Hive?
TIA,
Chuck Connell
Nuance R&D Data Team
Burlington, MA
RE: Problem replacing existing Hive file with modified copy
Posted by "Connell, Chuck" <Ch...@nuance.com>.
Gee thanks! That is great service.
Chuck
From: Bejoy Ks [mailto:bejoy_ks@yahoo.com]
Sent: Wednesday, July 25, 2012 12:04 PM
To: user@hive.apache.org
Subject: Re: Problem replacing existing Hive file with modified copy
The corresponding jira filed to track this bug is 'HIVE-3300' .
https://issues.apache.org/jira/browse/HIVE-3300
Regards
Bejoy KS
________________________________
From: Bejoy Ks <be...@yahoo.com>>
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Sent: Wednesday, July 25, 2012 9:28 PM
Subject: Re: Problem replacing existing Hive file with modified copy
Hi Connell
It looks like a bug in hive, I checked with hive 0.9 . If you are loading data from local fs to hive tables using 'LOAD DATA LOCAL INPATH' and if a file with the same name exists in the table's location then the new file will be suffixed by *_copy_1.
But if we do the 'LOAD DATA INPATH' for a file in hdfs then there is no rename happening but just a move task is getting triggered. Since a file with same name exists in same hdfs location, hadoop fs move operation throws an error.
I'll file a jira for this issue and update the same here.
Regards
Bejoy KS
________________________________
From: "Connell, Chuck" <Ch...@nuance.com>>
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Sent: Wednesday, July 25, 2012 8:51 PM
Subject: Problem replacing existing Hive file with modified copy
I created a Hive table that consists of two files, names1.txt and names2.txt. The table works correctly and answers all queries etc.
I want to REPLACE names2.txt with a modified version. I copied the new version of names2.txt to the /tmp/input folder within HDFS. Then I tried the command:
hive -e "load data inpath '/tmp/input/names2.txt' into table names"
I get an error: "Failed with exception null. Exception error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask"
Is this just a bug in this version (hadoop-hive-0.7.1+42.43-2.noarch, via CDH3 distro)?
Is my syntax correct to update a file in Hive?
TIA,
Chuck Connell
Nuance R&D Data Team
Burlington, MA
Re: Problem replacing existing Hive file with modified copy
Posted by Bejoy Ks <be...@yahoo.com>.
The corresponding jira filed to track this bug is 'HIVE-3300' .
https://issues.apache.org/jira/browse/HIVE-3300
Regards
Bejoy KS
________________________________
From: Bejoy Ks <be...@yahoo.com>
To: "user@hive.apache.org" <us...@hive.apache.org>
Sent: Wednesday, July 25, 2012 9:28 PM
Subject: Re: Problem replacing existing Hive file with modified copy
Hi Connell
It looks like a bug in hive, I checked with hive 0.9 . If you are loading data from local fs to hive tables using 'LOAD DATA LOCAL INPATH' and if a file with the same name exists in the table's location then the new file will be suffixed by *_copy_1.
But if we do the 'LOAD DATA INPATH' for a file in hdfs then there is no rename happening but just a move task is getting triggered. Since a file with same name exists in same hdfs location, hadoop fs move operation throws an error.
I'll file a jira for this issue and update the same here.
Regards
Bejoy KS
________________________________
From: "Connell, Chuck" <Ch...@nuance.com>
To: "user@hive.apache.org" <us...@hive.apache.org>
Sent: Wednesday, July 25, 2012 8:51 PM
Subject: Problem replacing existing Hive file with modified copy
I created a Hive table that consists of two files, names1.txt and names2.txt. The table works correctly and answers all queries etc.
I want to REPLACE names2.txt with a modified version. I copied the new version of names2.txt to the /tmp/input folder within HDFS. Then I tried the command:
hive -e "load data inpath '/tmp/input/names2.txt' into table names"
I get an error: "Failed with exception null. Exception error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask"
Is this just a bug in this version (hadoop-hive-0.7.1+42.43-2.noarch, via CDH3 distro)?
Is my syntax correct to update a file in Hive?
TIA,
Chuck Connell
Nuance R&D Data Team
Burlington, MA
Re: Problem replacing existing Hive file with modified copy
Posted by Bejoy Ks <be...@yahoo.com>.
Hi Connell
It looks like a bug in hive, I checked with hive 0.9 . If you are loading data from local fs to hive tables using 'LOAD DATA LOCAL INPATH' and if a file with the same name exists in the table's location then the new file will be suffixed by *_copy_1.
But if we do the 'LOAD DATA INPATH' for a file in hdfs then there is no rename happening but just a move task is getting triggered. Since a file with same name exists in same hdfs location, hadoop fs move operation throws an error.
I'll file a jira for this issue and update the same here.
Regards
Bejoy KS
________________________________
From: "Connell, Chuck" <Ch...@nuance.com>
To: "user@hive.apache.org" <us...@hive.apache.org>
Sent: Wednesday, July 25, 2012 8:51 PM
Subject: Problem replacing existing Hive file with modified copy
I created a Hive table that consists of two files, names1.txt and names2.txt. The table works correctly and answers all queries etc.
I want to REPLACE names2.txt with a modified version. I copied the new version of names2.txt to the /tmp/input folder within HDFS. Then I tried the command:
hive -e "load data inpath '/tmp/input/names2.txt' into table names"
I get an error: "Failed with exception null. Exception error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask"
Is this just a bug in this version (hadoop-hive-0.7.1+42.43-2.noarch, via CDH3 distro)?
Is my syntax correct to update a file in Hive?
TIA,
Chuck Connell
Nuance R&D Data Team
Burlington, MA