You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2011/01/04 05:03:46 UTC
[jira] Commented: (MAPREDUCE-1820) InputSampler does not create a
deep copy of the key object when creating a sample, which causes problems
with some formats like SequenceFile
[ https://issues.apache.org/jira/browse/MAPREDUCE-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977100#action_12977100 ]
Hudson commented on MAPREDUCE-1820:
-----------------------------------
Integrated in HBase-TRUNK #1701 (See [https://hudson.apache.org/hudson/job/HBase-TRUNK/1701/])
HBASE-3392. Update backport of InputSampler to reflect MAPREDUCE-1820
> InputSampler does not create a deep copy of the key object when creating a sample, which causes problems with some formats like SequenceFile<Text,Text>
> -------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1820
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1820
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Alex Kozlov
> Assignee: Alex Kozlov
> Fix For: 0.22.0
>
> Attachments: M1820-4.patch, M1820-5.patch, MAPREDUCE-1820-2.patch, MAPREDUCE-1820-3.patch, MAPREDUCE-1820.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> I tried to use the InputSampler on a SequenceFile<Text,Text> and found that it comes up with duplicate keys in the sample. The problem was tracked down to the fact that the Text object returned from the reader is essentially a wrapper pointing to a byte array, which changes as the sequence file reader progresses. There was also a bug in that the reader should be initialized before the use. The am attaching a patch that fixes both of the issues. --Alex K
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.