You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Pradeep Kamath (JIRA)" <ji...@apache.org> on 2010/01/29 21:32:34 UTC
[jira] Created: (PIG-1213) Schema serialization is broken
Schema serialization is broken
------------------------------
Key: PIG-1213
URL: https://issues.apache.org/jira/browse/PIG-1213
Project: Pig
Issue Type: Bug
Affects Versions: 0.6.0
Reporter: Pradeep Kamath
Assignee: Pradeep Kamath
Fix For: 0.6.0
Consider a udf which needs to know the schema of its input in the backend while executing. To achieve this, the udf needs to store the schema into the UDFContext. Internally the UDFContext will serialize the schema into the jobconf. However this currently is broken and gives a Serialization exception
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1213) Schema serialization is broken
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-1213:
--------------------------------
Status: Patch Available (was: Open)
> Schema serialization is broken
> ------------------------------
>
> Key: PIG-1213
> URL: https://issues.apache.org/jira/browse/PIG-1213
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Pradeep Kamath
> Assignee: Pradeep Kamath
> Fix For: 0.6.0
>
> Attachments: PIG-1213.patch
>
>
> Consider a udf which needs to know the schema of its input in the backend while executing. To achieve this, the udf needs to store the schema into the UDFContext. Internally the UDFContext will serialize the schema into the jobconf. However this currently is broken and gives a Serialization exception
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1213) Schema serialization is broken
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-1213:
--------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Patch was committed to trunk and branch-0.6 on 01 Feb 2010
> Schema serialization is broken
> ------------------------------
>
> Key: PIG-1213
> URL: https://issues.apache.org/jira/browse/PIG-1213
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Pradeep Kamath
> Assignee: Pradeep Kamath
> Fix For: 0.6.0
>
> Attachments: PIG-1213.patch
>
>
> Consider a udf which needs to know the schema of its input in the backend while executing. To achieve this, the udf needs to store the schema into the UDFContext. Internally the UDFContext will serialize the schema into the jobconf. However this currently is broken and gives a Serialization exception
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1213) Schema serialization is broken
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806621#action_12806621 ]
Hadoop QA commented on PIG-1213:
--------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12431826/PIG-1213.patch
against trunk revision 904241.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/193/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/193/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/193/console
This message is automatically generated.
> Schema serialization is broken
> ------------------------------
>
> Key: PIG-1213
> URL: https://issues.apache.org/jira/browse/PIG-1213
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Pradeep Kamath
> Assignee: Pradeep Kamath
> Fix For: 0.6.0
>
> Attachments: PIG-1213.patch
>
>
> Consider a udf which needs to know the schema of its input in the backend while executing. To achieve this, the udf needs to store the schema into the UDFContext. Internally the UDFContext will serialize the schema into the jobconf. However this currently is broken and gives a Serialization exception
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1213) Schema serialization is broken
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12806540#action_12806540 ]
Daniel Dai commented on PIG-1213:
---------------------------------
+1. Please commit once hudson reviewed.
> Schema serialization is broken
> ------------------------------
>
> Key: PIG-1213
> URL: https://issues.apache.org/jira/browse/PIG-1213
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Pradeep Kamath
> Assignee: Pradeep Kamath
> Fix For: 0.6.0
>
> Attachments: PIG-1213.patch
>
>
> Consider a udf which needs to know the schema of its input in the backend while executing. To achieve this, the udf needs to store the schema into the UDFContext. Internally the UDFContext will serialize the schema into the jobconf. However this currently is broken and gives a Serialization exception
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1213) Schema serialization is broken
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pradeep Kamath updated PIG-1213:
--------------------------------
Attachment: PIG-1213.patch
Attached patch addresses the issue - a couple of data structures in Schema are only used by the front-end query planning code to figure out lineage to handle casts of bytearrays. These data structures don't need to be serialized to the backend. Hence in the patch they are marked transient which fixes the issue.
> Schema serialization is broken
> ------------------------------
>
> Key: PIG-1213
> URL: https://issues.apache.org/jira/browse/PIG-1213
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Pradeep Kamath
> Assignee: Pradeep Kamath
> Fix For: 0.6.0
>
> Attachments: PIG-1213.patch
>
>
> Consider a udf which needs to know the schema of its input in the backend while executing. To achieve this, the udf needs to store the schema into the UDFContext. Internally the UDFContext will serialize the schema into the jobconf. However this currently is broken and gives a Serialization exception
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.