You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Zheng Shao (JIRA)" <ji...@apache.org> on 2009/05/19 00:50:46 UTC
[jira] Created: (HIVE-495) Join of a table with ThriftSerDe of
complex columns will fail
Join of a table with ThriftSerDe of complex columns will fail
-------------------------------------------------------------
Key: HIVE-495
URL: https://issues.apache.org/jira/browse/HIVE-495
Project: Hadoop Hive
Issue Type: Bug
Reporter: Zheng Shao
Assignee: Zheng Shao
ThriftSerDe exposes its fields using java class names. LazySerDe requires fully-blown map/struct/array format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-495) Join of a table with ThriftSerDe of
complex columns will fail
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715619#action_12715619 ]
Namit Jain commented on HIVE-495:
---------------------------------
My bad: did not apply the patch correctly
Am merging right now
> Join of a table with ThriftSerDe of complex columns will fail
> -------------------------------------------------------------
>
> Key: HIVE-495
> URL: https://issues.apache.org/jira/browse/HIVE-495
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: complex.seq, HIVE-495.1.patch
>
>
> ThriftSerDe exposes its fields using java class names. LazySerDe requires fully-blown map/struct/array format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-495) Join of a table with ThriftSerDe of
complex columns will fail
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710611#action_12710611 ]
Zheng Shao commented on HIVE-495:
---------------------------------
There are 3 approaches to fix this:
1. Use fully-blown map/struct/array format everywhere. In this solution, users will see the types of complex columns fully-blown instead of a java class, which may not be wanted.
2. Use 2 type strings: internal type string, and display type string. This will make sure users will see types of complex columns as before, but it makes the code more convoluted.
3. Make our TypeInfoParser capable of parsing Java class names. If the TypeInfoParser encounters a Java class name, it will use the ReflectionObjectInspector to get the fully-blown TypeInfo structure.
I am going to take approach 3.
> Join of a table with ThriftSerDe of complex columns will fail
> -------------------------------------------------------------
>
> Key: HIVE-495
> URL: https://issues.apache.org/jira/browse/HIVE-495
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Zheng Shao
> Assignee: Zheng Shao
>
> ThriftSerDe exposes its fields using java class names. LazySerDe requires fully-blown map/struct/array format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-495) Join of a table with ThriftSerDe of
complex columns will fail
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710626#action_12710626 ]
Zheng Shao commented on HIVE-495:
---------------------------------
More debugging shows that the main problem is that we were not allowing "_" in the names of the fields.
> Join of a table with ThriftSerDe of complex columns will fail
> -------------------------------------------------------------
>
> Key: HIVE-495
> URL: https://issues.apache.org/jira/browse/HIVE-495
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Zheng Shao
> Assignee: Zheng Shao
>
> ThriftSerDe exposes its fields using java class names. LazySerDe requires fully-blown map/struct/array format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-495) Join of a table with ThriftSerDe of
complex columns will fail
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-495:
----------------------------
Attachment: complex.seq
This is a binary file so I have to attach it separately.
> Join of a table with ThriftSerDe of complex columns will fail
> -------------------------------------------------------------
>
> Key: HIVE-495
> URL: https://issues.apache.org/jira/browse/HIVE-495
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: complex.seq, HIVE-495.1.patch
>
>
> ThriftSerDe exposes its fields using java class names. LazySerDe requires fully-blown map/struct/array format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-495) Join of a table with ThriftSerDe of
complex columns will fail
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-495:
----------------------------
Resolution: Fixed
Fix Version/s: 0.4.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed. Thanks Zheng
> Join of a table with ThriftSerDe of complex columns will fail
> -------------------------------------------------------------
>
> Key: HIVE-495
> URL: https://issues.apache.org/jira/browse/HIVE-495
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Fix For: 0.4.0
>
> Attachments: complex.seq, HIVE-495.1.patch
>
>
> ThriftSerDe exposes its fields using java class names. LazySerDe requires fully-blown map/struct/array format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-495) Join of a table with ThriftSerDe of
complex columns will fail
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-495:
----------------------------
Attachment: HIVE-495.1.patch
This patch fixes the bugs.
> Join of a table with ThriftSerDe of complex columns will fail
> -------------------------------------------------------------
>
> Key: HIVE-495
> URL: https://issues.apache.org/jira/browse/HIVE-495
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-495.1.patch
>
>
> ThriftSerDe exposes its fields using java class names. LazySerDe requires fully-blown map/struct/array format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-495) Join of a table with ThriftSerDe of
complex columns will fail
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715603#action_12715603 ]
Namit Jain commented on HIVE-495:
---------------------------------
The patch looks good - but there were failures when I ran the test
input5, input17 and join_thrift
> Join of a table with ThriftSerDe of complex columns will fail
> -------------------------------------------------------------
>
> Key: HIVE-495
> URL: https://issues.apache.org/jira/browse/HIVE-495
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: complex.seq, HIVE-495.1.patch
>
>
> ThriftSerDe exposes its fields using java class names. LazySerDe requires fully-blown map/struct/array format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-495) Join of a table with ThriftSerDe of
complex columns will fail
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-495:
----------------------------
Status: Patch Available (was: Open)
> Join of a table with ThriftSerDe of complex columns will fail
> -------------------------------------------------------------
>
> Key: HIVE-495
> URL: https://issues.apache.org/jira/browse/HIVE-495
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: HIVE-495.1.patch
>
>
> ThriftSerDe exposes its fields using java class names. LazySerDe requires fully-blown map/struct/array format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.