You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Stanley Xu <we...@gmail.com> on 2011/05/05 17:47:04 UTC
Is there any way I could force recover a HBase table that has missing blocks.
Sorry, missing type the title. Should be "Is there any way I could force
recover a HBase table that has missing blocks."
On Thu, May 5, 2011 at 10:27 PM, Stanley Xu <we...@gmail.com> wrote:
> Dear all,
>
> We were doing some network migration on our system and made some mistakes
> in the operation. So for some of our hbase data, we have some block missed,
> like the following by a fsck command output(in the end part of the mail). I
> am wondering if we could just ignore the missing blocks(let's say we just
> lost part of the data) but still keep the whole table available? Because our
> backup didn't cover all the data in the hbase table. If we could ignore the
> missing blocks, and do an overwrite with the backup data, the data we lost
> is trivial.
>
> If we have to drop the table and recover from the file backup, we might
> lose some of the column family that we didn't back up in our backup storage
> system. Thanks in advance.
>
> Status: CORRUPT
> Total size: 27674055256 B
> Total dirs: 1777
> Total files: 3094
> Total blocks (validated): 3170 (avg. block size 8729985 B)
> ********************************
> CORRUPT FILES: 22
> MISSING BLOCKS: 22
> MISSING SIZE: 176158662 B
> CORRUPT BLOCKS: 22
> ********************************
> Minimally replicated blocks: 3148 (99.30599 %)
> Over-replicated blocks: 0 (0.0 %)
> Under-replicated blocks: 0 (0.0 %)
> Mis-replicated blocks: 0 (0.0 %)
> Default replication factor: 3
> Average block replication: 2.9681387
> Corrupt blocks: 22
> Missing replicas: 0 (0.0 %)
> Number of data-nodes: 37
> Number of racks: 1
>
>
> The filesystem under path '/hbase/URLTag' is CORRUPT
>
>
>
> Best wishes,
> Stanley Xu
>
>
Re: Is there any way I could force recover a HBase table that has
missing blocks.
Posted by Stanley Xu <we...@gmail.com>.
Thanks J-D. Seems the only way is running a fsck -delete.
On Fri, May 6, 2011 at 12:50 AM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> If you can't find the missing blocks you'll have to delete the corrupted
> files.
>
> J-D
>
> On Thu, May 5, 2011 at 8:47 AM, Stanley Xu <we...@gmail.com> wrote:
> > Sorry, missing type the title. Should be "Is there any way I could force
> > recover a HBase table that has missing blocks."
> > On Thu, May 5, 2011 at 10:27 PM, Stanley Xu <we...@gmail.com> wrote:
> >
> >> Dear all,
> >>
> >> We were doing some network migration on our system and made some
> mistakes
> >> in the operation. So for some of our hbase data, we have some block
> missed,
> >> like the following by a fsck command output(in the end part of the
> mail). I
> >> am wondering if we could just ignore the missing blocks(let's say we
> just
> >> lost part of the data) but still keep the whole table available? Because
> our
> >> backup didn't cover all the data in the hbase table. If we could ignore
> the
> >> missing blocks, and do an overwrite with the backup data, the data we
> lost
> >> is trivial.
> >>
> >> If we have to drop the table and recover from the file backup, we might
> >> lose some of the column family that we didn't back up in our backup
> storage
> >> system. Thanks in advance.
> >>
> >> Status: CORRUPT
> >> Total size: 27674055256 B
> >> Total dirs: 1777
> >> Total files: 3094
> >> Total blocks (validated): 3170 (avg. block size 8729985 B)
> >> ********************************
> >> CORRUPT FILES: 22
> >> MISSING BLOCKS: 22
> >> MISSING SIZE: 176158662 B
> >> CORRUPT BLOCKS: 22
> >> ********************************
> >> Minimally replicated blocks: 3148 (99.30599 %)
> >> Over-replicated blocks: 0 (0.0 %)
> >> Under-replicated blocks: 0 (0.0 %)
> >> Mis-replicated blocks: 0 (0.0 %)
> >> Default replication factor: 3
> >> Average block replication: 2.9681387
> >> Corrupt blocks: 22
> >> Missing replicas: 0 (0.0 %)
> >> Number of data-nodes: 37
> >> Number of racks: 1
> >>
> >>
> >> The filesystem under path '/hbase/URLTag' is CORRUPT
> >>
> >>
> >>
> >> Best wishes,
> >> Stanley Xu
> >>
> >>
> >
>
Re: Is there any way I could force recover a HBase table that has
missing blocks.
Posted by Jean-Daniel Cryans <jd...@apache.org>.
If you can't find the missing blocks you'll have to delete the corrupted files.
J-D
On Thu, May 5, 2011 at 8:47 AM, Stanley Xu <we...@gmail.com> wrote:
> Sorry, missing type the title. Should be "Is there any way I could force
> recover a HBase table that has missing blocks."
> On Thu, May 5, 2011 at 10:27 PM, Stanley Xu <we...@gmail.com> wrote:
>
>> Dear all,
>>
>> We were doing some network migration on our system and made some mistakes
>> in the operation. So for some of our hbase data, we have some block missed,
>> like the following by a fsck command output(in the end part of the mail). I
>> am wondering if we could just ignore the missing blocks(let's say we just
>> lost part of the data) but still keep the whole table available? Because our
>> backup didn't cover all the data in the hbase table. If we could ignore the
>> missing blocks, and do an overwrite with the backup data, the data we lost
>> is trivial.
>>
>> If we have to drop the table and recover from the file backup, we might
>> lose some of the column family that we didn't back up in our backup storage
>> system. Thanks in advance.
>>
>> Status: CORRUPT
>> Total size: 27674055256 B
>> Total dirs: 1777
>> Total files: 3094
>> Total blocks (validated): 3170 (avg. block size 8729985 B)
>> ********************************
>> CORRUPT FILES: 22
>> MISSING BLOCKS: 22
>> MISSING SIZE: 176158662 B
>> CORRUPT BLOCKS: 22
>> ********************************
>> Minimally replicated blocks: 3148 (99.30599 %)
>> Over-replicated blocks: 0 (0.0 %)
>> Under-replicated blocks: 0 (0.0 %)
>> Mis-replicated blocks: 0 (0.0 %)
>> Default replication factor: 3
>> Average block replication: 2.9681387
>> Corrupt blocks: 22
>> Missing replicas: 0 (0.0 %)
>> Number of data-nodes: 37
>> Number of racks: 1
>>
>>
>> The filesystem under path '/hbase/URLTag' is CORRUPT
>>
>>
>>
>> Best wishes,
>> Stanley Xu
>>
>>
>