You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Harsh J Chouraria (JIRA)" <ji...@apache.org> on 2011/02/26 07:47:21 UTC
[jira] Commented: (MAPREDUCE-1360) Record skipping should work with
more serializations
[ https://issues.apache.org/jira/browse/MAPREDUCE-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12999728#comment-12999728 ]
Harsh J Chouraria commented on MAPREDUCE-1360:
----------------------------------------------
This sounded interesting for me to work upon. So here's some questions after a brief investigation:
I investigated what goes on inside of the SkippingRecordReader but couldn't find any area that gets Writable limited. The sequence files for skipped records are created with the Key/Value classes which are then used to load their acceptable serialization classes. Which part of the skipping framework is Writable limited now?
> Record skipping should work with more serializations
> ----------------------------------------------------
>
> Key: MAPREDUCE-1360
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1360
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Aaron Kimball
>
> Record skipping currently supports WritableSerialization, but cannot handle non-class-based serialization systems (e.g., AvroSerialization). The record skipping mechanism should be made compatible with the metadata-based serialization configuration.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira