You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by 김형준 <ji...@nhncorp.com> on 2006/10/17 03:22:40 UTC
A record(one text line) is laid across two blocks
I have a question.
If a record(one text line) is laid across two blocks, map function can't process that line
because the line from TextInputFormat is not complete line.
How do I resolve this problem?
Does HDFS support line formatted upload?(like DFSTextOutputStream),
Thank for your reply.
Re: A record(one text line) is laid across two blocks
Posted by Benjamin Reed <br...@yahoo-inc.com>.
Most InputFormats handle records that span blocks by always reading one
record into the next block and always skipping the first record of the
block unless it is reading the first block.
ben
김형준 wrote:
> I have a question.
>
> If a record(one text line) is laid across two blocks, map function can't process that line
> because the line from TextInputFormat is not complete line.
> How do I resolve this problem?
>
> Does HDFS support line formatted upload?(like DFSTextOutputStream),
>
> Thank for your reply.