You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hbase.apache.org by Michael Dagaev <mi...@gmail.com> on 2009/01/27 17:24:10 UTC

Question on HDFS corruption

Hi, all

      When we ran an Hbase cluster w/o upping open files limit,
the data nodes failed with "too many open files" exception.
As a result, the entire HDFS file system is corrupted now.

Does anybody know how to fix a corrupted HDFS?

Thank you for your cooperation,
M.

Re: Question on HDFS corruption

Posted by Michael Dagaev <mi...@gmail.com>.

>> As a result, the entire HDFS file system is corrupted now.
>>
>
> Tell more Michael.  Are blocks missing?

Yes.

> How does the corruption manifest?

Hadoop fsck prints out a lot of missing blocks
and says that the file system is corrupted.
The region servers failed to start because of that.

Now I upped the number of open files,
removed the file system and ran the test again.

(it was a test environment).

M.

Re: Question on HDFS corruption

Posted by stack <st...@duboce.net>.

Michael Dagaev wrote:
> Hi, all
>
>       When we ran an Hbase cluster w/o upping open files limit,
> the data nodes failed with "too many open files" exception.
>   
Its too late now, I know, but in the Requirements section, its explicit 
about the need to up file limits: 
http://hadoop.apache.org/hbase/docs/r0.19.0/api/index.html

> As a result, the entire HDFS file system is corrupted now.
>   

Tell more Michael.  Are blocks missing?  How does the corruption manifest?

St.Ack