You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jun Young Kim <ju...@gmail.com> on 2011/02/18 03:29:42 UTC

I got errors from hdfs about DataStreamer Exceptions.

hi, all.

I got errors from hdfs.

2011-02-18 11:21:29[WARN ][DFSOutputStream.java]run()(519) : DataStreamer Exception: java.io.IOException: Unable to create new block.
	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:832)
	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)

2011-02-18 11:21:29[WARN ][DFSOutputStream.java]setupPipelineForAppendOrRecovery()(730) : Could not get block locations. Source file "/user/test/51/output/ehshop00newsvc-r-00000" - Aborting...
2011-02-18 11:21:29[WARN ][Child.java]main()(234) : Exception running child : java.io.EOFException
	at java.io.DataInputStream.readShort(DataInputStream.java:298)
	at org.apache.hadoop.hdfs.protocol.DataTransferProtocol$Status.read(DataTransferProtocol.java:113)
	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:881)
	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:820)
	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)

2011-02-18 11:21:29[INFO ][Task.java]taskCleanup()(996) : Runnning cleanup for the task



I think this one is also not different error.

org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: blk_-2325764274016776017_8292 file=/user/test/51/input/kids.txt

         at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:559)

         at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:367)

         at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:514)

         at java.io.DataInputStream.read(DataInputStream.java:83)

         at org.apache.hadoop.util.LineReader.readLine(LineReader.java:138)

         at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:149)

         at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:465)

         at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)

         at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:90)

         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)


--> I've checked the file '/user/test/51/input/kids.txt ', but, there is 
not strange ones. this file is healthy.

Does anybody know about this error?
How could I fix this one?

thanks.

-- 
Junyoung Kim (juneng603@gmail.com)

Re: I got errors from hdfs about DataStreamer Exceptions.

Posted by 茅旭峰 <m9...@gmail.com>.
I double checked the cluster, all of the disk have plenty of free space.
I could not put any data into the cluster. The cluster summary shows

==
Cluster Summary***1987209 files and directories, 1993327 blocks = 3980536
total. Heap Size is 4.2 GB / 5.56 GB (75%)
*
Configured Capacity : 120.88 TB DFS Used : 49.55 TB Non DFS Used :
6.12 TBDFS Remaining:65.21 TBDFS Used%:40.99 %DFS Remaining%:53.95 %
Live
Nodes <http://cdn-hadoop-01:50070/dfsnodelist.jsp?whatNodes=LIVE> : 14 Dead
Nodes <http://cdn-hadoop-01:50070/dfsnodelist.jsp?whatNodes=DEAD> : 0
==

and hadoop fsck / shows good
==
Status: HEALTHY
 Total size:    17900052452875 B
 Total dirs:    35613
 Total files:   1951594 (Files currently being written: 1)
 Total blocks (validated):      1992835 (avg. block size 8982204 B)
 Minimally replicated blocks:   1992835 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       5 (2.5089885E-4 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     3.0000422
 Corrupt blocks:                0
 Missing replicas:              21 (3.5125346E-4 %)
 Number of data-nodes:          14
 Number of racks:               1


The filesystem under path '/' is HEALTHY

==

Any suggestions? Thanks!

On Sat, Apr 16, 2011 at 9:48 PM, Sonal Goyal <so...@gmail.com> wrote:

> Hi,
>
> What is the cluster free disk space and current usage?
>
> Thanks and Regards,
> Sonal
> <https://github.com/sonalgoyal/hiho>Hadoop ETL and Data Integration<https://github.com/sonalgoyal/hiho>
> Nube Technologies <http://www.nubetech.co>
>
> <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
>
>
> On Sat, Apr 16, 2011 at 9:18 AM, 茅旭峰 <m9...@gmail.com> wrote:
>
>> Hi Jun Young,
>>
>> It looks like I'm also running into the same issue you've got.
>> Have you found more info? I googled
>> 'java.io.IOException: Unable to create new block.', some guys
>> suggested that it might be running out of fd, but it's not my case.
>> I verified the fds in use by lsof and ls /proc/pid/fd.
>> Do you have any hints?
>>
>> Thanks!
>>
>> Mao Xu-Feng
>>
>> On Fri, Feb 18, 2011 at 10:51 AM, Jun Young Kim <ju...@gmail.com>
>> wrote:
>>
>> > hi, harsh.
>> > you're always giving a response very quickly. ;)
>> >
>> > I am using a version 0.21.0 now.
>> > before asking about this problem, I've checked already file system
>> healthy.
>> >
>> > $> hadoop fsck /
>> > .....
>> > .....
>> > Status: HEALTHY
>> >  Total size:    24231595038 B
>> >  Total dirs:    43818
>> >  Total files:   41193 (Files currently being written: 2178)
>> >  Total blocks (validated):      40941 (avg. block size 591866 B) (Total
>> > open file blocks (not validated): 224)
>> >  Minimally replicated blocks:   40941 (100.0 %)
>> >  Over-replicated blocks:        1 (0.0024425392 %)
>> >  Under-replicated blocks:       2 (0.0048850784 %)
>> >  Mis-replicated blocks:         0 (0.0 %)
>> >  Default replication factor:    2
>> >  Average block replication:     2.1106226
>> >  Corrupt blocks:                0
>> >  Missing replicas:              4 (0.00462904 %)
>> >  Number of data-nodes:          8
>> >  Number of racks:               1
>> >
>> > The filesystem under path '/' is HEALTHY
>> >
>> > additionally, I found a little different error. here it is.
>> >
>> > java.io.IOException: Bad connect ack with firstBadLink as
>> > 10.25.241.107:50010 at
>> >
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:889)
>> > at
>> >
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:820)
>> > at
>> >
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)
>> >
>> >
>> > here is my execution environment.
>> >
>> > average job count : 20
>> > max map capacity : 128
>> > max reduce capacity : 128
>> > avg/slot per node : 32
>> >
>> > avg input file size per job : 200M ~ 1G
>> >
>> >
>> > thanks.
>> >
>> > Junyoung Kim (juneng603@gmail.com)
>> >
>> >
>> > On 02/18/2011 11:43 AM, Harsh J wrote:
>> >
>> >> You may want to check your HDFS health stat via 'fsck'
>> >> (http://namenode/fsck or `hadoop fsck`). There may be a few corrupt
>> >> files or bad DNs.
>> >>
>> >> Would also be good to know what exact version of Hadoop you're running.
>> >>
>> >> On Fri, Feb 18, 2011 at 7:59 AM, Jun Young Kim<ju...@gmail.com>
>> >>  wrote:
>> >>
>> >>> hi, all.
>> >>>
>> >>> I got errors from hdfs.
>> >>>
>> >>> 2011-02-18 11:21:29[WARN ][DFSOutputStream.java]run()(519) :
>> DataStreamer
>> >>> Exception: java.io.IOException: Unable to create new block.
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:832)
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)
>> >>>
>> >>> 2011-02-18 11:21:29[WARN
>> >>> ][DFSOutputStream.java]setupPipelineForAppendOrRecovery()(730) : Could
>> >>> not
>> >>> get block locations. Source file
>> >>> "/user/test/51/output/ehshop00newsvc-r-00000" - Aborting...
>> >>> 2011-02-18 11:21:29[WARN ][Child.java]main()(234) : Exception running
>> >>> child
>> >>> : java.io.EOFException
>> >>>        at java.io.DataInputStream.readShort(DataInputStream.java:298)
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.hdfs.protocol.DataTransferProtocol$Status.read(DataTransferProtocol.java:113)
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:881)
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:820)
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)
>> >>>
>> >>> 2011-02-18 11:21:29[INFO ][Task.java]taskCleanup()(996) : Runnning
>> >>> cleanup
>> >>> for the task
>> >>>
>> >>>
>> >>>
>> >>> I think this one is also not different error.
>> >>>
>> >>> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
>> >>> blk_-2325764274016776017_8292 file=/user/test/51/input/kids.txt
>> >>>
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:559)
>> >>>
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:367)
>> >>>
>> >>>        at
>> >>> org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:514)
>> >>>
>> >>>        at java.io.DataInputStream.read(DataInputStream.java:83)
>> >>>
>> >>>        at
>> org.apache.hadoop.util.LineReader.readLine(LineReader.java:138)
>> >>>
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:149)
>> >>>
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:465)
>> >>>
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>> >>>
>> >>>        at
>> >>>
>> >>>
>> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:90)
>> >>>
>> >>>        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
>> >>>
>> >>>
>> >>> -->  I've checked the file '/user/test/51/input/kids.txt ', but, there
>> is
>> >>> not
>> >>> strange ones. this file is healthy.
>> >>>
>> >>> Does anybody know about this error?
>> >>> How could I fix this one?
>> >>>
>> >>> thanks.
>> >>>
>> >>> --
>> >>> Junyoung Kim (juneng603@gmail.com)
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>>
>
>

Re: I got errors from hdfs about DataStreamer Exceptions.

Posted by 茅旭峰 <m9...@gmail.com>.
Hi Jun Young,

It looks like I'm also running into the same issue you've got.
Have you found more info? I googled
'java.io.IOException: Unable to create new block.', some guys
suggested that it might be running out of fd, but it's not my case.
I verified the fds in use by lsof and ls /proc/pid/fd.
Do you have any hints?

Thanks!

Mao Xu-Feng

On Fri, Feb 18, 2011 at 10:51 AM, Jun Young Kim <ju...@gmail.com> wrote:

> hi, harsh.
> you're always giving a response very quickly. ;)
>
> I am using a version 0.21.0 now.
> before asking about this problem, I've checked already file system healthy.
>
> $> hadoop fsck /
> .....
> .....
> Status: HEALTHY
>  Total size:    24231595038 B
>  Total dirs:    43818
>  Total files:   41193 (Files currently being written: 2178)
>  Total blocks (validated):      40941 (avg. block size 591866 B) (Total
> open file blocks (not validated): 224)
>  Minimally replicated blocks:   40941 (100.0 %)
>  Over-replicated blocks:        1 (0.0024425392 %)
>  Under-replicated blocks:       2 (0.0048850784 %)
>  Mis-replicated blocks:         0 (0.0 %)
>  Default replication factor:    2
>  Average block replication:     2.1106226
>  Corrupt blocks:                0
>  Missing replicas:              4 (0.00462904 %)
>  Number of data-nodes:          8
>  Number of racks:               1
>
> The filesystem under path '/' is HEALTHY
>
> additionally, I found a little different error. here it is.
>
> java.io.IOException: Bad connect ack with firstBadLink as
> 10.25.241.107:50010 at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:889)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:820)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)
>
>
> here is my execution environment.
>
> average job count : 20
> max map capacity : 128
> max reduce capacity : 128
> avg/slot per node : 32
>
> avg input file size per job : 200M ~ 1G
>
>
> thanks.
>
> Junyoung Kim (juneng603@gmail.com)
>
>
> On 02/18/2011 11:43 AM, Harsh J wrote:
>
>> You may want to check your HDFS health stat via 'fsck'
>> (http://namenode/fsck or `hadoop fsck`). There may be a few corrupt
>> files or bad DNs.
>>
>> Would also be good to know what exact version of Hadoop you're running.
>>
>> On Fri, Feb 18, 2011 at 7:59 AM, Jun Young Kim<ju...@gmail.com>
>>  wrote:
>>
>>> hi, all.
>>>
>>> I got errors from hdfs.
>>>
>>> 2011-02-18 11:21:29[WARN ][DFSOutputStream.java]run()(519) : DataStreamer
>>> Exception: java.io.IOException: Unable to create new block.
>>>        at
>>>
>>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:832)
>>>        at
>>>
>>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)
>>>
>>> 2011-02-18 11:21:29[WARN
>>> ][DFSOutputStream.java]setupPipelineForAppendOrRecovery()(730) : Could
>>> not
>>> get block locations. Source file
>>> "/user/test/51/output/ehshop00newsvc-r-00000" - Aborting...
>>> 2011-02-18 11:21:29[WARN ][Child.java]main()(234) : Exception running
>>> child
>>> : java.io.EOFException
>>>        at java.io.DataInputStream.readShort(DataInputStream.java:298)
>>>        at
>>>
>>> org.apache.hadoop.hdfs.protocol.DataTransferProtocol$Status.read(DataTransferProtocol.java:113)
>>>        at
>>>
>>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:881)
>>>        at
>>>
>>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:820)
>>>        at
>>>
>>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)
>>>
>>> 2011-02-18 11:21:29[INFO ][Task.java]taskCleanup()(996) : Runnning
>>> cleanup
>>> for the task
>>>
>>>
>>>
>>> I think this one is also not different error.
>>>
>>> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
>>> blk_-2325764274016776017_8292 file=/user/test/51/input/kids.txt
>>>
>>>        at
>>>
>>> org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:559)
>>>
>>>        at
>>>
>>> org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:367)
>>>
>>>        at
>>> org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:514)
>>>
>>>        at java.io.DataInputStream.read(DataInputStream.java:83)
>>>
>>>        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:138)
>>>
>>>        at
>>>
>>> org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:149)
>>>
>>>        at
>>>
>>> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:465)
>>>
>>>        at
>>>
>>> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>>>
>>>        at
>>>
>>> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:90)
>>>
>>>        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
>>>
>>>
>>> -->  I've checked the file '/user/test/51/input/kids.txt ', but, there is
>>> not
>>> strange ones. this file is healthy.
>>>
>>> Does anybody know about this error?
>>> How could I fix this one?
>>>
>>> thanks.
>>>
>>> --
>>> Junyoung Kim (juneng603@gmail.com)
>>>
>>>
>>>
>>
>>

Re: I got errors from hdfs about DataStreamer Exceptions.

Posted by Jun Young Kim <ju...@gmail.com>.
hi, harsh.
you're always giving a response very quickly. ;)

I am using a version 0.21.0 now.
before asking about this problem, I've checked already file system healthy.

$> hadoop fsck /
.....
.....
Status: HEALTHY
  Total size:    24231595038 B
  Total dirs:    43818
  Total files:   41193 (Files currently being written: 2178)
  Total blocks (validated):      40941 (avg. block size 591866 B) (Total 
open file blocks (not validated): 224)
  Minimally replicated blocks:   40941 (100.0 %)
  Over-replicated blocks:        1 (0.0024425392 %)
  Under-replicated blocks:       2 (0.0048850784 %)
  Mis-replicated blocks:         0 (0.0 %)
  Default replication factor:    2
  Average block replication:     2.1106226
  Corrupt blocks:                0
  Missing replicas:              4 (0.00462904 %)
  Number of data-nodes:          8
  Number of racks:               1

The filesystem under path '/' is HEALTHY

additionally, I found a little different error. here it is.

java.io.IOException: Bad connect ack with firstBadLink as 
10.25.241.107:50010 at 
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:889) 
at 
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:820) 
at 
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)


here is my execution environment.

average job count : 20
max map capacity : 128
max reduce capacity : 128
avg/slot per node : 32

avg input file size per job : 200M ~ 1G

thanks.

Junyoung Kim (juneng603@gmail.com)


On 02/18/2011 11:43 AM, Harsh J wrote:
> You may want to check your HDFS health stat via 'fsck'
> (http://namenode/fsck or `hadoop fsck`). There may be a few corrupt
> files or bad DNs.
>
> Would also be good to know what exact version of Hadoop you're running.
>
> On Fri, Feb 18, 2011 at 7:59 AM, Jun Young Kim<ju...@gmail.com>  wrote:
>> hi, all.
>>
>> I got errors from hdfs.
>>
>> 2011-02-18 11:21:29[WARN ][DFSOutputStream.java]run()(519) : DataStreamer
>> Exception: java.io.IOException: Unable to create new block.
>>         at
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:832)
>>         at
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)
>>
>> 2011-02-18 11:21:29[WARN
>> ][DFSOutputStream.java]setupPipelineForAppendOrRecovery()(730) : Could not
>> get block locations. Source file
>> "/user/test/51/output/ehshop00newsvc-r-00000" - Aborting...
>> 2011-02-18 11:21:29[WARN ][Child.java]main()(234) : Exception running child
>> : java.io.EOFException
>>         at java.io.DataInputStream.readShort(DataInputStream.java:298)
>>         at
>> org.apache.hadoop.hdfs.protocol.DataTransferProtocol$Status.read(DataTransferProtocol.java:113)
>>         at
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:881)
>>         at
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:820)
>>         at
>> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)
>>
>> 2011-02-18 11:21:29[INFO ][Task.java]taskCleanup()(996) : Runnning cleanup
>> for the task
>>
>>
>>
>> I think this one is also not different error.
>>
>> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
>> blk_-2325764274016776017_8292 file=/user/test/51/input/kids.txt
>>
>>         at
>> org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:559)
>>
>>         at
>> org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:367)
>>
>>         at
>> org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:514)
>>
>>         at java.io.DataInputStream.read(DataInputStream.java:83)
>>
>>         at org.apache.hadoop.util.LineReader.readLine(LineReader.java:138)
>>
>>         at
>> org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:149)
>>
>>         at
>> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:465)
>>
>>         at
>> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>>
>>         at
>> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:90)
>>
>>         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
>>
>>
>> -->  I've checked the file '/user/test/51/input/kids.txt ', but, there is not
>> strange ones. this file is healthy.
>>
>> Does anybody know about this error?
>> How could I fix this one?
>>
>> thanks.
>>
>> --
>> Junyoung Kim (juneng603@gmail.com)
>>
>>
>
>

Re: I got errors from hdfs about DataStreamer Exceptions.

Posted by Harsh J <qw...@gmail.com>.
You may want to check your HDFS health stat via 'fsck'
(http://namenode/fsck or `hadoop fsck`). There may be a few corrupt
files or bad DNs.

Would also be good to know what exact version of Hadoop you're running.

On Fri, Feb 18, 2011 at 7:59 AM, Jun Young Kim <ju...@gmail.com> wrote:
> hi, all.
>
> I got errors from hdfs.
>
> 2011-02-18 11:21:29[WARN ][DFSOutputStream.java]run()(519) : DataStreamer
> Exception: java.io.IOException: Unable to create new block.
>        at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:832)
>        at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)
>
> 2011-02-18 11:21:29[WARN
> ][DFSOutputStream.java]setupPipelineForAppendOrRecovery()(730) : Could not
> get block locations. Source file
> "/user/test/51/output/ehshop00newsvc-r-00000" - Aborting...
> 2011-02-18 11:21:29[WARN ][Child.java]main()(234) : Exception running child
> : java.io.EOFException
>        at java.io.DataInputStream.readShort(DataInputStream.java:298)
>        at
> org.apache.hadoop.hdfs.protocol.DataTransferProtocol$Status.read(DataTransferProtocol.java:113)
>        at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:881)
>        at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:820)
>        at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427)
>
> 2011-02-18 11:21:29[INFO ][Task.java]taskCleanup()(996) : Runnning cleanup
> for the task
>
>
>
> I think this one is also not different error.
>
> org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
> blk_-2325764274016776017_8292 file=/user/test/51/input/kids.txt
>
>        at
> org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:559)
>
>        at
> org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:367)
>
>        at
> org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:514)
>
>        at java.io.DataInputStream.read(DataInputStream.java:83)
>
>        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:138)
>
>        at
> org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:149)
>
>        at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:465)
>
>        at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>
>        at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:90)
>
>        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
>
>
> --> I've checked the file '/user/test/51/input/kids.txt ', but, there is not
> strange ones. this file is healthy.
>
> Does anybody know about this error?
> How could I fix this one?
>
> thanks.
>
> --
> Junyoung Kim (juneng603@gmail.com)
>
>



-- 
Harsh J
www.harshj.com