You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by SingoWong <si...@gmail.com> on 2010/12/24 23:44:13 UTC
The Warning message in DFS home page about missing blocks, and cannot
use the data
Hi,
In my DFS home page, display the message below
WARNING : There are about 7179 missing blocks. Please check the log or run
fsck.
who can tell me what this message and how to fix than? bcoz so far i cannt
get and analyse the file in hadoop.
And i check the log file,
2010-12-25 06:38:41,787 INFO
org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
succeeded for blk_6082125532449737116_6174
2010-12-25 06:39:13,788 INFO
org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
succeeded for blk_2326255566552911767_4485
2010-12-25 06:39:45,788 INFO
org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
succeeded for blk_-5212470058352205240_50675
2010-12-25 06:40:17,985 INFO
org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
succeeded for blk_7053283143490404909_46108
2010-12-25 06:40:49,986 INFO
org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
succeeded for blk_1148971771233953883_4046
2010-12-25 06:41:21,987 INFO
org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
succeeded for blk_7357967897891404229_50302
2010-12-25 06:41:22,434 INFO
org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
succeeded for blk_2780273733665219859_32726
But it is start up about 4 days, who can help me? what time should the
verification and how can i fix the warning for missing blocks?
Happy X'max & Regards,
Singo
Re: The Warning message in DFS home page about missing blocks, and
cannot use the data
Posted by Stack <st...@duboce.net>.
If block is missing and all datanodes have checked in, then its gone.
Grep the block name in namednode to get sense of the blocks history.
Can you figure what happened to it?
St.Ack
On Sun, Dec 26, 2010 at 2:47 AM, SingoWong <si...@gmail.com> wrote:
> but how to slove? fsck only to show which paths are corrupt, but how can i
> fix or still the missing blocks?
>
> Regards,
> Singo
>
> On Sat, Dec 25, 2010 at 6:49 AM, Ryan Rawson <ry...@gmail.com> wrote:
>
>> Missing blocks = missing datanodes (or datanode/data directory
>> corruption). You can run bin/hadoop fsck / to see what was affected.
>>
>> I generally run in to this when not all datanodes have joined the namenode.
>>
>> 'fsck' will show you which paths are "corrupt" (aka missing replicas).
>>
>> Good luck!
>> -ryan
>>
>> On Fri, Dec 24, 2010 at 2:44 PM, SingoWong <si...@gmail.com> wrote:
>> > Hi,
>> >
>> > In my DFS home page, display the message below
>> > WARNING : There are about 7179 missing blocks. Please check the log or
>> run
>> > fsck.
>> >
>> > who can tell me what this message and how to fix than? bcoz so far i
>> cannt
>> > get and analyse the file in hadoop.
>> >
>> > And i check the log file,
>> > 2010-12-25 06:38:41,787 INFO
>> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
>> > succeeded for blk_6082125532449737116_6174
>> > 2010-12-25 06:39:13,788 INFO
>> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
>> > succeeded for blk_2326255566552911767_4485
>> > 2010-12-25 06:39:45,788 INFO
>> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
>> > succeeded for blk_-5212470058352205240_50675
>> > 2010-12-25 06:40:17,985 INFO
>> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
>> > succeeded for blk_7053283143490404909_46108
>> > 2010-12-25 06:40:49,986 INFO
>> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
>> > succeeded for blk_1148971771233953883_4046
>> > 2010-12-25 06:41:21,987 INFO
>> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
>> > succeeded for blk_7357967897891404229_50302
>> > 2010-12-25 06:41:22,434 INFO
>> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
>> > succeeded for blk_2780273733665219859_32726
>> >
>> > But it is start up about 4 days, who can help me? what time should the
>> > verification and how can i fix the warning for missing blocks?
>> >
>> > Happy X'max & Regards,
>> > Singo
>> >
>>
>
Re: The Warning message in DFS home page about missing blocks, and
cannot use the data
Posted by SingoWong <si...@gmail.com>.
but how to slove? fsck only to show which paths are corrupt, but how can i
fix or still the missing blocks?
Regards,
Singo
On Sat, Dec 25, 2010 at 6:49 AM, Ryan Rawson <ry...@gmail.com> wrote:
> Missing blocks = missing datanodes (or datanode/data directory
> corruption). You can run bin/hadoop fsck / to see what was affected.
>
> I generally run in to this when not all datanodes have joined the namenode.
>
> 'fsck' will show you which paths are "corrupt" (aka missing replicas).
>
> Good luck!
> -ryan
>
> On Fri, Dec 24, 2010 at 2:44 PM, SingoWong <si...@gmail.com> wrote:
> > Hi,
> >
> > In my DFS home page, display the message below
> > WARNING : There are about 7179 missing blocks. Please check the log or
> run
> > fsck.
> >
> > who can tell me what this message and how to fix than? bcoz so far i
> cannt
> > get and analyse the file in hadoop.
> >
> > And i check the log file,
> > 2010-12-25 06:38:41,787 INFO
> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> > succeeded for blk_6082125532449737116_6174
> > 2010-12-25 06:39:13,788 INFO
> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> > succeeded for blk_2326255566552911767_4485
> > 2010-12-25 06:39:45,788 INFO
> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> > succeeded for blk_-5212470058352205240_50675
> > 2010-12-25 06:40:17,985 INFO
> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> > succeeded for blk_7053283143490404909_46108
> > 2010-12-25 06:40:49,986 INFO
> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> > succeeded for blk_1148971771233953883_4046
> > 2010-12-25 06:41:21,987 INFO
> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> > succeeded for blk_7357967897891404229_50302
> > 2010-12-25 06:41:22,434 INFO
> > org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> > succeeded for blk_2780273733665219859_32726
> >
> > But it is start up about 4 days, who can help me? what time should the
> > verification and how can i fix the warning for missing blocks?
> >
> > Happy X'max & Regards,
> > Singo
> >
>
Re: The Warning message in DFS home page about missing blocks, and
cannot use the data
Posted by Ryan Rawson <ry...@gmail.com>.
Missing blocks = missing datanodes (or datanode/data directory
corruption). You can run bin/hadoop fsck / to see what was affected.
I generally run in to this when not all datanodes have joined the namenode.
'fsck' will show you which paths are "corrupt" (aka missing replicas).
Good luck!
-ryan
On Fri, Dec 24, 2010 at 2:44 PM, SingoWong <si...@gmail.com> wrote:
> Hi,
>
> In my DFS home page, display the message below
> WARNING : There are about 7179 missing blocks. Please check the log or run
> fsck.
>
> who can tell me what this message and how to fix than? bcoz so far i cannt
> get and analyse the file in hadoop.
>
> And i check the log file,
> 2010-12-25 06:38:41,787 INFO
> org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> succeeded for blk_6082125532449737116_6174
> 2010-12-25 06:39:13,788 INFO
> org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> succeeded for blk_2326255566552911767_4485
> 2010-12-25 06:39:45,788 INFO
> org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> succeeded for blk_-5212470058352205240_50675
> 2010-12-25 06:40:17,985 INFO
> org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> succeeded for blk_7053283143490404909_46108
> 2010-12-25 06:40:49,986 INFO
> org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> succeeded for blk_1148971771233953883_4046
> 2010-12-25 06:41:21,987 INFO
> org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> succeeded for blk_7357967897891404229_50302
> 2010-12-25 06:41:22,434 INFO
> org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> succeeded for blk_2780273733665219859_32726
>
> But it is start up about 4 days, who can help me? what time should the
> verification and how can i fix the warning for missing blocks?
>
> Happy X'max & Regards,
> Singo
>