You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Chinna Rao Lalam (JIRA)" <ji...@apache.org> on 2011/02/08 16:34:57 UTC
[jira] Created: (HIVE-1975) "insert overwrite directory" Not able
to insert data with multi level directory path
"insert overwrite directory" Not able to insert data with multi level directory path
------------------------------------------------------------------------------------
Key: HIVE-1975
URL: https://issues.apache.org/jira/browse/HIVE-1975
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.5.0
Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
Below query execution is failed
Ex:
{noformat}
insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
{noformat}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103160#comment-13103160 ]
He Yongqiang commented on HIVE-1975:
------------------------------------
what's the use case here? the user can always first create the parent dir. But users misspell the dir name, they may not want the dirs created. Or worse, the data got loaded to some other place they not noticed.
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "Ovidiu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015309#comment-13015309 ]
Ovidiu commented on HIVE-1975:
------------------------------
Any update on this particular issue?
I am trying to do the same but using s3 storage and it seems hive can only create one new directory and not a full structure. Is this a limitation?
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1975) "insert overwrite directory" Not able
to insert data with multi level directory path
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-1975:
-----------------------------------
Attachment: HIVE-1975.1.patch
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1975) "insert overwrite directory" Not able
to insert data with multi level directory path
Posted by "Chinna Rao Lalam (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-1975:
-----------------------------------
Attachment: HIVE-1975.3.patch
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.3.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103799#comment-13103799 ]
jiraposter@reviews.apache.org commented on HIVE-1975:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1829/
-----------------------------------------------------------
Review request for hive and Yongqiang He.
Summary
-------
If insert into multilevel directory like '/HIVEFT25686/chinna/'
(insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j) is failing because "fs.rename(sourcePath, targetPath)" is failing rename to multilevel directories, so first created the target path.
This addresses bug HIVE-1975.
https://issues.apache.org/jira/browse/HIVE-1975
Diffs
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 1170044
Diff: https://reviews.apache.org/r/1829/diff
Testing
-------
Ran all testcases
Thanks,
chinna
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1975) "insert overwrite directory" Not able
to insert data with multi level directory path
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-1975:
-----------------------------------
Attachment: HIVE-1975.patch
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1975) "insert overwrite directory" Not able
to insert data with multi level directory path
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-1975:
-----------------------------------
Status: Patch Available (was: Open)
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1975) "insert overwrite directory" Not able
to insert data with multi level directory path
Posted by "Chinna Rao Lalam (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-1975:
-----------------------------------
Status: Patch Available (was: Open)
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103707#comment-13103707 ]
Chinna Rao Lalam commented on HIVE-1975:
----------------------------------------
Use case is if we want to insert into multilevel directory like '/HIVEFT25686/chinna/'
(insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j) this is failing because "fs.rename(sourcePath, targetPath)" is failing rename to multilevel directories, so first created the target path.
>> But users misspell the dir name, they may not want the dirs created.Or worse, the data got loaded to some other place they not noticed.
This one will happen in the normal scenario also what ever the location user specifies it will move the data to that location.
But as u said if the rename operation fails it should delete the created directory. I will update the patch with this case.
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1975) "insert overwrite directory" Not able
to insert data with multi level directory path
Posted by "Chinna Rao Lalam (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-1975:
-----------------------------------
Status: Patch Available (was: Open)
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.3.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015473#comment-13015473 ]
Edward Capriolo commented on HIVE-1975:
---------------------------------------
Are you asking if hive can export a partitioned table to a flat directory? please include more details in your report. Such as describe extended dept_j;
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1975) "insert overwrite directory" Not able
to insert data with multi level directory path
Posted by "Chinna Rao Lalam (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-1975:
-----------------------------------
Attachment: HIVE-1975.2.patch
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1975) "insert overwrite directory" Not able
to insert data with multi level directory path
Posted by "Chinna Rao Lalam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinna Rao Lalam updated HIVE-1975:
-----------------------------------
Status: Open (was: Patch Available)
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139115#comment-13139115 ]
Hudson commented on HIVE-1975:
------------------------------
Integrated in Hive-trunk-h0.21 #1042 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1042/])
HIVE-1975 "insert overwrite directory" Not able to insert data with multi level
directory path (Chinna Rao Lalam via namit)
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1190719
Files :
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/conf/hive-default.xml
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java
* /hive/trunk/ql/src/test/queries/clientpositive/input45.q
* /hive/trunk/ql/src/test/results/clientpositive/input45.q.out
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.3.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "Namit Jain (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138537#comment-13138537 ]
Namit Jain commented on HIVE-1975:
----------------------------------
+1
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.3.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1975) "insert overwrite directory" Not able
to insert data with multi level directory path
Posted by "Namit Jain (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1975:
-----------------------------
Status: Open (was: Patch Available)
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132830#comment-13132830 ]
jiraposter@reviews.apache.org commented on HIVE-1975:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1829/
-----------------------------------------------------------
(Updated 2011-10-21 17:09:08.230077)
Review request for hive and Yongqiang He.
Changes
-------
Reworked on the comments
Summary
-------
If insert into multilevel directory like '/HIVEFT25686/chinna/'
(insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j) is failing because "fs.rename(sourcePath, targetPath)" is failing rename to multilevel directories, so first created the target path.
This addresses bug HIVE-1975.
https://issues.apache.org/jira/browse/HIVE-1975
Diffs (updated)
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 1186875
Diff: https://reviews.apache.org/r/1829/diff
Testing
-------
Ran all testcases
Thanks,
chinna
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138380#comment-13138380 ]
jiraposter@reviews.apache.org commented on HIVE-1975:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1829/
-----------------------------------------------------------
(Updated 2011-10-28 13:53:07.086932)
Review request for hive and Yongqiang He.
Changes
-------
Addressed Namit & He Yongqiang comments
Summary
-------
If insert into multilevel directory like '/HIVEFT25686/chinna/'
(insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j) is failing because "fs.rename(sourcePath, targetPath)" is failing rename to multilevel directories, so first created the target path.
This addresses bug HIVE-1975.
https://issues.apache.org/jira/browse/HIVE-1975
Diffs (updated)
-----
trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1190202
trunk/conf/hive-default.xml 1190202
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 1190202
trunk/ql/src/test/queries/clientpositive/input45.q PRE-CREATION
trunk/ql/src/test/results/clientpositive/input45.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1829/diff
Testing
-------
Ran all testcases
Thanks,
chinna
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.3.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "He Yongqiang (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134548#comment-13134548 ]
He Yongqiang commented on HIVE-1975:
------------------------------------
it is good to add a conf for this, and by default disable it.
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1975) "insert overwrite directory" Not able
to insert data with multi level directory path
Posted by "Namit Jain (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1975:
-----------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed. Thanks Chinna
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.3.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "Namit Jain (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134265#comment-13134265 ]
Namit Jain commented on HIVE-1975:
----------------------------------
Some minor comments:
1. Can you add a testcase - use build as a temporary directory to add data ?
2. Add some comments explaining that the multi-level directory move is not atomic.
In your example, it is possible that /x/y/z/1/2/3 is created, but /x/y/z/1/2/4.
It is same as any multi-partition insert (dynamic insert), but would be good to
explicitly call it out.
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1975) "insert overwrite directory" Not
able to insert data with multi level directory path
Posted by "Chinna Rao Lalam (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138379#comment-13138379 ]
Chinna Rao Lalam commented on HIVE-1975:
----------------------------------------
Addressed Namit & He Yongqiang comments
> "insert overwrite directory" Not able to insert data with multi level directory path
> ------------------------------------------------------------------------------------
>
> Key: HIVE-1975
> URL: https://issues.apache.org/jira/browse/HIVE-1975
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Environment: Hadoop 0.20.1, Hive0.5.0 and SUSE Linux Enterprise Server 10 SP2 (i586) - Kernel 2.6.16.60-0.21-smp (5).
> Reporter: Chinna Rao Lalam
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-1975.1.patch, HIVE-1975.2.patch, HIVE-1975.3.patch, HIVE-1975.patch
>
>
> Below query execution is failed
> Ex:
> {noformat}
> insert overwrite directory '/HIVEFT25686/chinna/' select * from dept_j;
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira