You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Sushanth Sowmyan (JIRA)" <ji...@apache.org> on 2012/07/01 14:52:29 UTC
[jira] [Created] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Sushanth Sowmyan created HIVE-3221:
--------------------------------------
Summary: HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
Key: HIVE-3221
URL: https://issues.apache.org/jira/browse/HIVE-3221
Project: Hive
Issue Type: Bug
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404728#comment-13404728 ]
Sushanth Sowmyan commented on HIVE-3221:
----------------------------------------
Now getting a new arc error on starting anew:
--
tundra:hive sush$ arc diff --jira HIVE-3221
PHP Fatal error: Call to undefined method ArcanistGitAPI::amendGitHeadCommit() in /Users/sush/dev/hive.git/.arc_jira_lib/arcanist/ArcJIRAConfiguration.php on line 169
Fatal error: Call to undefined method ArcanistGitAPI::amendGitHeadCommit() in /Users/sush/dev/hive.git/.arc_jira_lib/arcanist/ArcJIRAConfiguration.php on line 169
--
Still, the patch attached has a unit test and is a fairly straightforward patch. Hopefully it can be reviewed easily.
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sushanth Sowmyan updated HIVE-3221:
-----------------------------------
Status: Patch Available (was: Open)
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408241#comment-13408241 ]
Edward Capriolo commented on HIVE-3221:
---------------------------------------
Also have you considered the column alias is _c not _col, we can possibly determine this from HiveConf
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan resolved HIVE-3221.
------------------------------------
Resolution: Fixed
Fix Version/s: 0.10.0
Committed to trunk. Thanks, Sushanth!
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419858#comment-13419858 ]
Ashutosh Chauhan commented on HIVE-3221:
----------------------------------------
@Ed,
There is a unit test in the patch which fails without the fix.
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409714#comment-13409714 ]
Sushanth Sowmyan commented on HIVE-3221:
----------------------------------------
I don't have a specific query that can trigger off the top of my head - looking at ql.parse.SemanticAnalyzer indicates that this problem could occur when formulating a union plan.
Essentially, reading "_col12" maps to "2" because it only reads the last character.
The sequence of events that led to my encountering this issue is in HCATALOG-436 when testing with the JsonSerDe (any named-column based format, really) and having more than 10 columns. I wanted to read an internal column name(as generated by a CTAS operation) and be able to figure out position and noticed that there are a pair of functions in HiveConf for exactly this purpose : getPositionFromInternalName and getColumnInternalName.
--
As to the case of having a column alias being "_c" and not "_col", HiveConf.java seems to indicate a hardcoded "_col" in getColumnInternalName, so I retained that. Do you have any cases where it's "_c" ?
Thanks!
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sushanth Sowmyan updated HIVE-3221:
-----------------------------------
Attachment: HIVE-3221.patch
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HIVE-3221)
HiveConf.getPositionFromInternalName does not support more than sinle digit
column numbers
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409339#comment-13409339 ]
Namit Jain edited comment on HIVE-3221 at 7/9/12 11:31 AM:
-----------------------------------------------------------
Can you address Edward's comments ?
was (Author: namit):
Can you address Sushanth's comments ?
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-3221:
-----------------------------
Status: Open (was: Patch Available)
Can you address Sushanth's comments ?
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419863#comment-13419863 ]
Hudson commented on HIVE-3221:
------------------------------
Integrated in Hive-trunk-h0.21 #1558 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1558/])
HIVE-3221 : HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers (Sushanth Sowmyan via Ashutosh Chauhan) (Revision 1364038)
Result = FAILURE
hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1364038
Files :
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419817#comment-13419817 ]
Edward Capriolo commented on HIVE-3221:
---------------------------------------
Ashutosh. I did not want to see this issue get committed without a positive or negative test. We have a fix but no test coverage.
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.10.0
>
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408236#comment-13408236 ]
Edward Capriolo commented on HIVE-3221:
---------------------------------------
Can you please give an example of a query that would cause this?
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418119#comment-13418119 ]
Ashutosh Chauhan commented on HIVE-3221:
----------------------------------------
+1. Looks good. Will commit if tests pass.
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Attachments: HIVE-3221.patch
>
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3221) HiveConf.getPositionFromInternalName
does not support more than sinle digit column numbers
Posted by "Sushanth Sowmyan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404725#comment-13404725 ]
Sushanth Sowmyan commented on HIVE-3221:
----------------------------------------
Attaching patch since arc diff isn't working for me
--
tundra:hive sush$ arc diff --jira HIVE-3221
Exception
Parse Exception: Expected a hunk header, like 'Index: /path/to/file.ext' (svn), 'Property changes on: /path/to/file.ext' (svn properties), 'commit 59bcc3ad6775562f845953cf01624225' (git show), 'diff --git' (git diff), or '--- filename' (unified diff).
>>> 1
--
> HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers
> ------------------------------------------------------------------------------------------
>
> Key: HIVE-3221
> URL: https://issues.apache.org/jira/browse/HIVE-3221
> Project: Hive
> Issue Type: Bug
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
>
> For positions above 9, HiveConf.getPositionFromInternalName only looks at the last digit, and thus, causes collisions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira