You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Siying Dong (JIRA)" <ji...@apache.org> on 2011/05/26 23:22:47 UTC
[jira] [Created] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Dynamic Partitioning Failing because of characters not supported globStatus
---------------------------------------------------------------------------
Key: HIVE-2186
URL: https://issues.apache.org/jira/browse/HIVE-2186
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Siying Dong
Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045249#comment-13045249 ]
Hudson commented on HIVE-2186:
------------------------------
Integrated in Hive-trunk-h0.21 #764 (See [https://builds.apache.org/job/Hive-trunk-h0.21/764/])
HIVE-2186. Escaping More Characters When Generating HDFS Path Name From Table Columns (Franklin Hu via Siying Dong)
sdong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1132778
Files :
* /hive/trunk/data/files/escapetest.txt
* /hive/trunk/ql/src/test/results/clientpositive/escape1.q.out
* /hive/trunk/ql/src/test/queries/clientpositive/escape1.q
* /hive/trunk/common/src/java/org/apache/hadoop/hive/common/FileUtils.java
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Attachments: hive-2186.1.patch, hive-2186.2.patch, hive-2186.3.patch, hive-2186.4.patch, hive-2186.5.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siying Dong updated HIVE-2186:
------------------------------
Resolution: Fixed
Release Note: Committed. Thanks Franklin.
Status: Resolved (was: Patch Available)
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Attachments: hive-2186.1.patch, hive-2186.2.patch, hive-2186.3.patch, hive-2186.4.patch, hive-2186.5.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2186:
------------------------------
Attachment: hive-2186.3.patch
Added DROP PARTITION to test
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Attachments: hive-2186.1.patch, hive-2186.2.patch, hive-2186.3.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2186:
---------------------------------
Fix Version/s: 0.8.0
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Fix For: 0.8.0
>
> Attachments: hive-2186.1.patch, hive-2186.2.patch, hive-2186.3.patch, hive-2186.4.patch, hive-2186.5.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2186:
------------------------------
Attachment: hive-2186.4.patch
show partitions after dropping
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Attachments: hive-2186.1.patch, hive-2186.2.patch, hive-2186.3.patch, hive-2186.4.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043235#comment-13043235 ]
Siying Dong commented on HIVE-2186:
-----------------------------------
You need to show partition after dropping partition to make sure dropping partition succeeded.
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Attachments: hive-2186.1.patch, hive-2186.2.patch, hive-2186.3.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041773#comment-13041773 ]
Siying Dong commented on HIVE-2186:
-----------------------------------
@Franklin, in your test case, can you also drop the partition ds=1 and show partitions again to make sure those partitions can be safely dropped?
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Attachments: hive-2186.1.patch, hive-2186.2.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2186:
------------------------------
Status: Patch Available (was: Open)
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Attachments: hive-2186.1.patch, hive-2186.2.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siying Dong reassigned HIVE-2186:
---------------------------------
Assignee: Franklin Hu
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2186:
------------------------------
Attachment: hive-2186.5.patch
fix uploaded binary files
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Attachments: hive-2186.1.patch, hive-2186.2.patch, hive-2186.3.patch, hive-2186.4.patch, hive-2186.5.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2186:
------------------------------
Attachment: hive-2186.1.patch
This patch adds to the list of illegal file path characters that may cause globStatus to break.
These include:
* ASCII characters 01-1F
* '[' open brace
Also included is a test for dynamic partitions with all standard ASCII characters (0-127).
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Attachments: hive-2186.1.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Zhenxiao Luo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225761#comment-13225761 ]
Zhenxiao Luo commented on HIVE-2186:
------------------------------------
Whoops, seems escape1.q.out is loaded as data, not ascii files. It is difficult to check diffs for new patches.
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Fix For: 0.8.0
>
> Attachments: hive-2186.1.patch, hive-2186.2.patch, hive-2186.3.patch, hive-2186.4.patch, hive-2186.5.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2186:
------------------------------
Attachment: hive-2186.2.patch
Added comments, additions to test file
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Attachments: hive-2186.1.patch, hive-2186.2.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2186) Dynamic Partitioning Failing because
of characters not supported globStatus
Posted by "Zhenxiao Luo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225715#comment-13225715 ]
Zhenxiao Luo commented on HIVE-2186:
------------------------------------
Seems '^' also needs to be escaped.
If add:
SELECT count(*) from escape_raw;
SELECT count(*) from escape1;
into escape1.q
escape1 only get 127 rows, while, escape_raw get 128 rows(note that escapetest.txt has 128 rows).
one row missing in escape1, which is the '^'.
This is to be fixed in HIVE-2856. Comments are appreciated.
Thanks,
Zhenxiao
> Dynamic Partitioning Failing because of characters not supported globStatus
> ---------------------------------------------------------------------------
>
> Key: HIVE-2186
> URL: https://issues.apache.org/jira/browse/HIVE-2186
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Siying Dong
> Assignee: Franklin Hu
> Fix For: 0.8.0
>
> Attachments: hive-2186.1.patch, hive-2186.2.patch, hive-2186.3.patch, hive-2186.4.patch, hive-2186.5.patch
>
>
> Some dynamic queries failed on the stage of loading partitions if dynamic partition columns contain special characters. We need to escape all of them.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira