You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ning Zhang (Created) (JIRA)" <ji...@apache.org> on 2011/10/20 23:19:10 UTC
[jira] [Created] (HIVE-2519) Dynamic partition insert should
enforce the order of the partition spec is the same as the one in schema
Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema
--------------------------------------------------------------------------------------------------------
Key: HIVE-2519
URL: https://issues.apache.org/jira/browse/HIVE-2519
Project: Hive
Issue Type: Bug
Reporter: Ning Zhang
Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
insert overwrite ... partition (p2="...", p1);
which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2519) Dynamic partition insert should
enforce the order of the partition spec is the same as the one in schema
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133835#comment-13133835 ]
jiraposter@reviews.apache.org commented on HIVE-2519:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2541/
-----------------------------------------------------------
(Updated 2011-10-24 05:25:21.441963)
Review request for hive.
Summary
-------
Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
insert overwrite ... partition (p2="...", p1);
which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
This addresses bug HIVE-2519.
https://issues.apache.org/jira/browse/HIVE-2519
Diffs (updated)
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1188024
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1188024
trunk/ql/src/test/queries/clientnegative/dyn_part4.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/dyn_part4.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/2541/diff
Testing
-------
ant test
Thanks,
Ning
> Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema
> --------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2519
> URL: https://issues.apache.org/jira/browse/HIVE-2519
> Project: Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.9.0
>
> Attachments: HIVE-2519.3.patch, HIVE-2519.patch
>
>
> Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
> insert overwrite ... partition (p2="...", p1);
> which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2519) Dynamic partition insert should
enforce the order of the partition spec is the same as the one in schema
Posted by "Ning Zhang (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang reassigned HIVE-2519:
--------------------------------
Assignee: Ning Zhang
> Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema
> --------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2519
> URL: https://issues.apache.org/jira/browse/HIVE-2519
> Project: Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
>
> Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
> insert overwrite ... partition (p2="...", p1);
> which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2519) Dynamic partition insert should
enforce the order of the partition spec is the same as the one in schema
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135314#comment-13135314 ]
Hudson commented on HIVE-2519:
------------------------------
Integrated in Hive-trunk-h0.21 #1032 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1032/])
HIVE-2519 Dynamic partition insert should enforce the order of the
partition spec is the same as the one in schema (Ning Zhang via namit)
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1188450
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java
* /hive/trunk/ql/src/test/queries/clientnegative/dyn_part4.q
* /hive/trunk/ql/src/test/results/clientnegative/dyn_part4.q.out
> Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema
> --------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2519
> URL: https://issues.apache.org/jira/browse/HIVE-2519
> Project: Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.9.0
>
> Attachments: HIVE-2519.3.patch, HIVE-2519.patch
>
>
> Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
> insert overwrite ... partition (p2="...", p1);
> which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2519) Dynamic partition insert should
enforce the order of the partition spec is the same as the one in schema
Posted by "Namit Jain (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134232#comment-13134232 ]
Namit Jain commented on HIVE-2519:
----------------------------------
+1
> Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema
> --------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2519
> URL: https://issues.apache.org/jira/browse/HIVE-2519
> Project: Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.9.0
>
> Attachments: HIVE-2519.3.patch, HIVE-2519.patch
>
>
> Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
> insert overwrite ... partition (p2="...", p1);
> which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2519) Dynamic partition insert should
enforce the order of the partition spec is the same as the one in schema
Posted by "Ning Zhang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-2519:
-----------------------------
Attachment: HIVE-2519.3.patch
cleaned code style
> Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema
> --------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2519
> URL: https://issues.apache.org/jira/browse/HIVE-2519
> Project: Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.9.0
>
> Attachments: HIVE-2519.3.patch, HIVE-2519.patch
>
>
> Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
> insert overwrite ... partition (p2="...", p1);
> which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2519) Dynamic partition insert should
enforce the order of the partition spec is the same as the one in schema
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133832#comment-13133832 ]
jiraposter@reviews.apache.org commented on HIVE-2519:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2541/
-----------------------------------------------------------
Review request for hive.
Summary
-------
Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
insert overwrite ... partition (p2="...", p1);
which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
This addresses bug HIVE-2519.
https://issues.apache.org/jira/browse/HIVE-2519
Diffs
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 1188024
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java 1188024
trunk/ql/src/test/queries/clientnegative/dyn_part4.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/dyn_part4.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/2541/diff
Testing
-------
ant test
Thanks,
Ning
> Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema
> --------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2519
> URL: https://issues.apache.org/jira/browse/HIVE-2519
> Project: Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.9.0
>
> Attachments: HIVE-2519.patch
>
>
> Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
> insert overwrite ... partition (p2="...", p1);
> which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2519) Dynamic partition insert should
enforce the order of the partition spec is the same as the one in schema
Posted by "Ning Zhang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-2519:
-----------------------------
Fix Version/s: 0.9.0
Status: Patch Available (was: Open)
> Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema
> --------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2519
> URL: https://issues.apache.org/jira/browse/HIVE-2519
> Project: Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.9.0
>
> Attachments: HIVE-2519.patch
>
>
> Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
> insert overwrite ... partition (p2="...", p1);
> which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2519) Dynamic partition insert should
enforce the order of the partition spec is the same as the one in schema
Posted by "Ning Zhang (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-2519:
-----------------------------
Attachment: HIVE-2519.patch
> Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema
> --------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2519
> URL: https://issues.apache.org/jira/browse/HIVE-2519
> Project: Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.9.0
>
> Attachments: HIVE-2519.patch
>
>
> Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
> insert overwrite ... partition (p2="...", p1);
> which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2519) Dynamic partition insert should
enforce the order of the partition spec is the same as the one in schema
Posted by "Namit Jain (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2519:
-----------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed. Thanks Ning
> Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema
> --------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2519
> URL: https://issues.apache.org/jira/browse/HIVE-2519
> Project: Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.9.0
>
> Attachments: HIVE-2519.3.patch, HIVE-2519.patch
>
>
> Suppose the table schema is (a string, b string) partitioned by (p1 string, p2 string), a dynamic partition insert is allowed to:
> insert overwrite ... partition (p2="...", p1);
> which will create the wrong HDFS directory structure such as /.../p2=.../p1=.... This is contradictory to the metastore's assumption of the HDFS directory structure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira