You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ning Zhang (JIRA)" <ji...@apache.org> on 2010/01/08 06:51:00 UTC
[jira] Created: (HIVE-1037) SerDe & ObjectInspector for
RowContainer mismatch with the input data
SerDe & ObjectInspector for RowContainer mismatch with the input data
---------------------------------------------------------------------
Key: HIVE-1037
URL: https://issues.apache.org/jira/browse/HIVE-1037
Project: Hadoop Hive
Issue Type: Bug
Reporter: Ning Zhang
Assignee: Ning Zhang
In CommonJoinOperator, RowContainer is created for each input table with the SerDe and ObjectInspector to serialize/deserialize that row to persistent storage. The serde/OI could be null in the case of the value columns are pruned by column pruner. An example query is
select count(1) from A join B on A.key=B.key;
Another case of mismatch is that the tableDesc was initialized at compile time before the column pruner take place. This could cause inconsistency in the SerDe/OI with the input data. This should be moved to execution time when the join operator is initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1037) SerDe & ObjectInspector for
RowContainer mismatch with the input data
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain resolved HIVE-1037.
------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Committed. Thanks Ning
> SerDe & ObjectInspector for RowContainer mismatch with the input data
> ---------------------------------------------------------------------
>
> Key: HIVE-1037
> URL: https://issues.apache.org/jira/browse/HIVE-1037
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Priority: Blocker
> Fix For: 0.5.0
>
> Attachments: HIVE-1037.patch
>
>
> In CommonJoinOperator, RowContainer is created for each input table with the SerDe and ObjectInspector to serialize/deserialize that row to persistent storage. The serde/OI could be null in the case of the value columns are pruned by column pruner. An example query is
> select count(1) from A join B on A.key=B.key;
> Another case of mismatch is that the tableDesc was initialized at compile time before the column pruner take place. This could cause inconsistency in the SerDe/OI with the input data. This should be moved to execution time when the join operator is initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1037) SerDe & ObjectInspector for
RowContainer mismatch with the input data
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798078#action_12798078 ]
Namit Jain commented on HIVE-1037:
----------------------------------
I agree - marked it as a blocker for 0.5
> SerDe & ObjectInspector for RowContainer mismatch with the input data
> ---------------------------------------------------------------------
>
> Key: HIVE-1037
> URL: https://issues.apache.org/jira/browse/HIVE-1037
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Priority: Blocker
> Fix For: 0.5.0
>
>
> In CommonJoinOperator, RowContainer is created for each input table with the SerDe and ObjectInspector to serialize/deserialize that row to persistent storage. The serde/OI could be null in the case of the value columns are pruned by column pruner. An example query is
> select count(1) from A join B on A.key=B.key;
> Another case of mismatch is that the tableDesc was initialized at compile time before the column pruner take place. This could cause inconsistency in the SerDe/OI with the input data. This should be moved to execution time when the join operator is initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1037) SerDe & ObjectInspector for
RowContainer mismatch with the input data
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1037:
-----------------------------
Priority: Blocker (was: Major)
> SerDe & ObjectInspector for RowContainer mismatch with the input data
> ---------------------------------------------------------------------
>
> Key: HIVE-1037
> URL: https://issues.apache.org/jira/browse/HIVE-1037
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Priority: Blocker
> Fix For: 0.5.0
>
>
> In CommonJoinOperator, RowContainer is created for each input table with the SerDe and ObjectInspector to serialize/deserialize that row to persistent storage. The serde/OI could be null in the case of the value columns are pruned by column pruner. An example query is
> select count(1) from A join B on A.key=B.key;
> Another case of mismatch is that the tableDesc was initialized at compile time before the column pruner take place. This could cause inconsistency in the SerDe/OI with the input data. This should be moved to execution time when the join operator is initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1037) SerDe & ObjectInspector for
RowContainer mismatch with the input data
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-1037:
-----------------------------
Attachment: HIVE-1037.patch
Uploading HIVE-1037.patch. Changes include:
1) moving initTableDesc from joinDesc to CommonJoinOperator to make it after column pruner.
2) some misc changes in RowContainer including better error reporting.
3) add a new unit test in join40.q
> SerDe & ObjectInspector for RowContainer mismatch with the input data
> ---------------------------------------------------------------------
>
> Key: HIVE-1037
> URL: https://issues.apache.org/jira/browse/HIVE-1037
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Priority: Blocker
> Fix For: 0.5.0
>
> Attachments: HIVE-1037.patch
>
>
> In CommonJoinOperator, RowContainer is created for each input table with the SerDe and ObjectInspector to serialize/deserialize that row to persistent storage. The serde/OI could be null in the case of the value columns are pruned by column pruner. An example query is
> select count(1) from A join B on A.key=B.key;
> Another case of mismatch is that the tableDesc was initialized at compile time before the column pruner take place. This could cause inconsistency in the SerDe/OI with the input data. This should be moved to execution time when the join operator is initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1037) SerDe & ObjectInspector for
RowContainer mismatch with the input data
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-1037:
-----------------------------
Fix Version/s: 0.5.0
I think this bug should be included in 0.5 since it may cause run time exception and wrong results.
> SerDe & ObjectInspector for RowContainer mismatch with the input data
> ---------------------------------------------------------------------
>
> Key: HIVE-1037
> URL: https://issues.apache.org/jira/browse/HIVE-1037
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Fix For: 0.5.0
>
>
> In CommonJoinOperator, RowContainer is created for each input table with the SerDe and ObjectInspector to serialize/deserialize that row to persistent storage. The serde/OI could be null in the case of the value columns are pruned by column pruner. An example query is
> select count(1) from A join B on A.key=B.key;
> Another case of mismatch is that the tableDesc was initialized at compile time before the column pruner take place. This could cause inconsistency in the SerDe/OI with the input data. This should be moved to execution time when the join operator is initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1037) SerDe & ObjectInspector for
RowContainer mismatch with the input data
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798187#action_12798187 ]
Namit Jain commented on HIVE-1037:
----------------------------------
+1
will commit if the tests pass
> SerDe & ObjectInspector for RowContainer mismatch with the input data
> ---------------------------------------------------------------------
>
> Key: HIVE-1037
> URL: https://issues.apache.org/jira/browse/HIVE-1037
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Priority: Blocker
> Fix For: 0.5.0
>
> Attachments: HIVE-1037.patch
>
>
> In CommonJoinOperator, RowContainer is created for each input table with the SerDe and ObjectInspector to serialize/deserialize that row to persistent storage. The serde/OI could be null in the case of the value columns are pruned by column pruner. An example query is
> select count(1) from A join B on A.key=B.key;
> Another case of mismatch is that the tableDesc was initialized at compile time before the column pruner take place. This could cause inconsistency in the SerDe/OI with the input data. This should be moved to execution time when the join operator is initialized.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.