A tool to rewrite corrupted HFile

[Andrey Elenskiy <an...@arista.com.INVALID>]

Hello,

Over years I have been dealing with corrupted hfiles by just removing them.
It always seemed wasteful to throw away the entire HFile even only if one
block is corrupted.

Do you happen to know if a tool to rewrite the hfile with omitting/skipping
corrupt blocks already exists? If not, would such a tool be of use to the
community? The goal of such a tool is to try and recover as much data as
possible. We can always drop data on the hbase block boundaries as well.

Andrey

Re: A tool to rewrite corrupted HFile

[Andrey Elenskiy <an...@arista.com.INVALID>]

Hey, opened a jira issue here:
https://issues.apache.org/jira/browse/HBASE-24920

Thanks!
Andrey

On Wed, Aug 19, 2020 at 7:14 PM Sean Busbey <bu...@apache.org> wrote:

> I think that would be a useful tool. It would also fit well into the
> project's hbase-operator-tools repo.
>
> Mind filing a jira to set some initial goals? Or look for suggestions on
> initial goal over on dev@hbase?
>
> On Wed, Aug 19, 2020, 18:10 Andrey Elenskiy
> <an...@arista.com.invalid> wrote:
>
> > Hello,
> >
> > Over years I have been dealing with corrupted hfiles by just removing
> them.
> > It always seemed wasteful to throw away the entire HFile even only if one
> > block is corrupted.
> >
> > Do you happen to know if a tool to rewrite the hfile with
> omitting/skipping
> > corrupt blocks already exists? If not, would such a tool be of use to the
> > community? The goal of such a tool is to try and recover as much data as
> > possible. We can always drop data on the hbase block boundaries as well.
> >
> > Andrey
> >
>

Re: A tool to rewrite corrupted HFile

[Sean Busbey <bu...@apache.org>]

I think that would be a useful tool. It would also fit well into the
project's hbase-operator-tools repo.

Mind filing a jira to set some initial goals? Or look for suggestions on
initial goal over on dev@hbase?

On Wed, Aug 19, 2020, 18:10 Andrey Elenskiy
<an...@arista.com.invalid> wrote:

> Hello,
>
> Over years I have been dealing with corrupted hfiles by just removing them.
> It always seemed wasteful to throw away the entire HFile even only if one
> block is corrupted.
>
> Do you happen to know if a tool to rewrite the hfile with omitting/skipping
> corrupt blocks already exists? If not, would such a tool be of use to the
> community? The goal of such a tool is to try and recover as much data as
> possible. We can always drop data on the hbase block boundaries as well.
>
> Andrey
>