You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Aaron Kimball (JIRA)" <ji...@apache.org> on 2009/12/16 03:14:18 UTC
[jira] Created: (HADOOP-6443) Serialization classes accept invalid
metadata
Serialization classes accept invalid metadata
---------------------------------------------
Key: HADOOP-6443
URL: https://issues.apache.org/jira/browse/HADOOP-6443
Project: Hadoop Common
Issue Type: Improvement
Components: io
Reporter: Aaron Kimball
Assignee: Aaron Kimball
The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6443) Serialization classes accept invalid
metadata
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6443:
----------------------------------
Status: Open (was: Patch Available)
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6443) Serialization classes accept invalid
metadata
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6443:
----------------------------------
Status: Patch Available (was: Open)
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6443) Serialization classes accept invalid
metadata
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-6443:
------------------------------
Resolution: Fixed
Fix Version/s: 0.22.0
Status: Resolved (was: Patch Available)
I've just committed this. Thanks Aaron!
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.3.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6443) Serialization classes accept invalid
metadata
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6443:
----------------------------------
Status: Patch Available (was: Open)
Submitting straightforward patch for this; this was written in conjunction with HADOOP-6323 but was split into a separate patch in my workflow. This code is tested via the serialization instantiations in MAPREDUCE-1126 (e.g., TestJavaSerialization, TestAvroSerialization, etc in the MAPREDUCE project).
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6443) Serialization classes accept invalid
metadata
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6443:
----------------------------------
Attachment: HADOOP-6443.3.patch
factored.
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.3.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6443) Serialization classes accept invalid
metadata
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6443:
----------------------------------
Status: Open (was: Patch Available)
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.3.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6443) Serialization classes accept invalid
metadata
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6443:
----------------------------------
Attachment: HADOOP-6443.2.patch
Added some unit test cases for this.
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6443) Serialization classes accept
invalid metadata
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796622#action_12796622 ]
Hudson commented on HADOOP-6443:
--------------------------------
Integrated in Hadoop-Common-trunk #209 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/209/])
. Serialization classes accept invalid metadata. Contributed by Aaron Kimball.
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.3.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6443) Serialization classes accept invalid
metadata
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6443:
----------------------------------
Status: Patch Available (was: Open)
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.3.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6443) Serialization classes accept invalid
metadata
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6443:
----------------------------------
Attachment: HADOOP-6443.patch
patch providing this behavior.
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6443) Serialization classes accept
invalid metadata
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791131#action_12791131 ]
Aaron Kimball commented on HADOOP-6443:
---------------------------------------
The accept methods of the serializers check whether SERIALIZATION_CLASS equals the current class name. If so, they accept unconditionally, even though they contain further (unreachable) logic to ensure that they contain the requisite metadata (e.g., an avro schema or a class name).
The check should work in the opposite direction: if the user specifies SERIALIZATION_CLASS and it does not equal the current class name, then the serialization should reject the metadata. If the SERIALIZATION_CLASS equals the current class name, or is simply unset, then the serialization should inspect the remainder of the metadata to ensure that it meets the criteria associated with the current serialization.
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6443) Serialization classes accept
invalid metadata
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795721#action_12795721 ]
Hadoop QA commented on HADOOP-6443:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12429238/HADOOP-6443.3.patch
against trunk revision 894706.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/246/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/246/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/246/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/246/console
This message is automatically generated.
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.3.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6443) Serialization classes accept
invalid metadata
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796448#action_12796448 ]
Hudson commented on HADOOP-6443:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #132 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/132/])
. Serialization classes accept invalid metadata. Contributed by Aaron Kimball.
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.3.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6443) Serialization classes accept
invalid metadata
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791138#action_12791138 ]
Hadoop QA commented on HADOOP-6443:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12428120/HADOOP-6443.patch
against trunk revision 890964.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/30/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/30/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/30/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/30/console
This message is automatically generated.
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6443) Serialization classes accept
invalid metadata
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791645#action_12791645 ]
Hadoop QA commented on HADOOP-6443:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12428232/HADOOP-6443.2.patch
against trunk revision 891132.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/212/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/212/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/212/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/212/console
This message is automatically generated.
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6443) Serialization classes accept
invalid metadata
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792741#action_12792741 ]
Tom White commented on HADOOP-6443:
-----------------------------------
+1
A nit: the logic for checking for SERIALIZATION_KEY is repeated in three places. Can you factor it out into a method?
> Serialization classes accept invalid metadata
> ---------------------------------------------
>
> Key: HADOOP-6443
> URL: https://issues.apache.org/jira/browse/HADOOP-6443
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6443.2.patch, HADOOP-6443.patch
>
>
> The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.