You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Bejoy KS (JIRA)" <ji...@apache.org> on 2012/07/25 18:03:33 UTC
[jira] [Created] (HIVE-3300) LOAD DATA INPATH fails if a hdfs file
with same name is added to table
Bejoy KS created HIVE-3300:
------------------------------
Summary: LOAD DATA INPATH fails if a hdfs file with same name is added to table
Key: HIVE-3300
URL: https://issues.apache.org/jira/browse/HIVE-3300
Project: Hive
Issue Type: Bug
Components: Import/Export
Affects Versions: 0.9.0
Environment: ubuntu linux, hadoop 1.0.3, hive 0.9
Reporter: Bejoy KS
If we are loading data from local fs to hive tables using 'LOAD DATA LOCAL INPATH' and if a file with the same name exists in the table's location then the new file will be suffixed by *_copy_1.
But if we do the 'LOAD DATA INPATH' for a file in hdfs then there is no rename happening but just a move task is getting triggered. Since a file with same name exists in same hdfs location, hadoop fs move operation throws an error.
hive> LOAD DATA INPATH '/userdata/bejoy/site.txt' INTO TABLE test.site;
Loading data to table test.site
Failed with exception null
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
hive>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3300) LOAD DATA INPATH fails if a hdfs file
with same name is added to table
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-3300:
------------------------
Status: Patch Available (was: Open)
> LOAD DATA INPATH fails if a hdfs file with same name is added to table
> ----------------------------------------------------------------------
>
> Key: HIVE-3300
> URL: https://issues.apache.org/jira/browse/HIVE-3300
> Project: Hive
> Issue Type: Bug
> Components: Import/Export
> Affects Versions: 0.10.0
> Environment: ubuntu linux, hadoop 1.0.3, hive 0.9
> Reporter: Bejoy KS
> Assignee: Navis
> Attachments: HIVE-3300.1.patch.txt
>
>
> If we are loading data from local fs to hive tables using 'LOAD DATA LOCAL INPATH' and if a file with the same name exists in the table's location then the new file will be suffixed by *_copy_1.
> But if we do the 'LOAD DATA INPATH' for a file in hdfs then there is no rename happening but just a move task is getting triggered. Since a file with same name exists in same hdfs location, hadoop fs move operation throws an error.
> hive> LOAD DATA INPATH '/userdata/bejoy/site.txt' INTO TABLE test.site;
> Loading data to table test.site
> Failed with exception null
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
> hive>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3300) LOAD DATA INPATH fails if a hdfs file
with same name is added to table
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-3300:
------------------------
Affects Version/s: (was: 0.9.0)
0.10.0
Status: Patch Available (was: Open)
https://reviews.facebook.net/D4383
> LOAD DATA INPATH fails if a hdfs file with same name is added to table
> ----------------------------------------------------------------------
>
> Key: HIVE-3300
> URL: https://issues.apache.org/jira/browse/HIVE-3300
> Project: Hive
> Issue Type: Bug
> Components: Import/Export
> Affects Versions: 0.10.0
> Environment: ubuntu linux, hadoop 1.0.3, hive 0.9
> Reporter: Bejoy KS
> Assignee: Navis
>
> If we are loading data from local fs to hive tables using 'LOAD DATA LOCAL INPATH' and if a file with the same name exists in the table's location then the new file will be suffixed by *_copy_1.
> But if we do the 'LOAD DATA INPATH' for a file in hdfs then there is no rename happening but just a move task is getting triggered. Since a file with same name exists in same hdfs location, hadoop fs move operation throws an error.
> hive> LOAD DATA INPATH '/userdata/bejoy/site.txt' INTO TABLE test.site;
> Loading data to table test.site
> Failed with exception null
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
> hive>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3300) LOAD DATA INPATH fails if a hdfs file
with same name is added to table
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-3300:
------------------------
Status: Open (was: Patch Available)
I think attached test is not appropriate for this usecase.
> LOAD DATA INPATH fails if a hdfs file with same name is added to table
> ----------------------------------------------------------------------
>
> Key: HIVE-3300
> URL: https://issues.apache.org/jira/browse/HIVE-3300
> Project: Hive
> Issue Type: Bug
> Components: Import/Export
> Affects Versions: 0.10.0
> Environment: ubuntu linux, hadoop 1.0.3, hive 0.9
> Reporter: Bejoy KS
> Assignee: Navis
> Attachments: HIVE-3300.1.patch.txt
>
>
> If we are loading data from local fs to hive tables using 'LOAD DATA LOCAL INPATH' and if a file with the same name exists in the table's location then the new file will be suffixed by *_copy_1.
> But if we do the 'LOAD DATA INPATH' for a file in hdfs then there is no rename happening but just a move task is getting triggered. Since a file with same name exists in same hdfs location, hadoop fs move operation throws an error.
> hive> LOAD DATA INPATH '/userdata/bejoy/site.txt' INTO TABLE test.site;
> Loading data to table test.site
> Failed with exception null
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
> hive>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3300) LOAD DATA INPATH fails if a hdfs file
with same name is added to table
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-3300:
------------------------
Attachment: HIVE-3300.1.patch.txt
> LOAD DATA INPATH fails if a hdfs file with same name is added to table
> ----------------------------------------------------------------------
>
> Key: HIVE-3300
> URL: https://issues.apache.org/jira/browse/HIVE-3300
> Project: Hive
> Issue Type: Bug
> Components: Import/Export
> Affects Versions: 0.10.0
> Environment: ubuntu linux, hadoop 1.0.3, hive 0.9
> Reporter: Bejoy KS
> Assignee: Navis
> Attachments: HIVE-3300.1.patch.txt
>
>
> If we are loading data from local fs to hive tables using 'LOAD DATA LOCAL INPATH' and if a file with the same name exists in the table's location then the new file will be suffixed by *_copy_1.
> But if we do the 'LOAD DATA INPATH' for a file in hdfs then there is no rename happening but just a move task is getting triggered. Since a file with same name exists in same hdfs location, hadoop fs move operation throws an error.
> hive> LOAD DATA INPATH '/userdata/bejoy/site.txt' INTO TABLE test.site;
> Loading data to table test.site
> Failed with exception null
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
> hive>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3300) LOAD DATA INPATH fails if a hdfs file
with same name is added to table
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis reassigned HIVE-3300:
---------------------------
Assignee: Navis
> LOAD DATA INPATH fails if a hdfs file with same name is added to table
> ----------------------------------------------------------------------
>
> Key: HIVE-3300
> URL: https://issues.apache.org/jira/browse/HIVE-3300
> Project: Hive
> Issue Type: Bug
> Components: Import/Export
> Affects Versions: 0.9.0
> Environment: ubuntu linux, hadoop 1.0.3, hive 0.9
> Reporter: Bejoy KS
> Assignee: Navis
>
> If we are loading data from local fs to hive tables using 'LOAD DATA LOCAL INPATH' and if a file with the same name exists in the table's location then the new file will be suffixed by *_copy_1.
> But if we do the 'LOAD DATA INPATH' for a file in hdfs then there is no rename happening but just a move task is getting triggered. Since a file with same name exists in same hdfs location, hadoop fs move operation throws an error.
> hive> LOAD DATA INPATH '/userdata/bejoy/site.txt' INTO TABLE test.site;
> Loading data to table test.site
> Failed with exception null
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
> hive>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira