Issues when splitting regions

[jiangwen w <wj...@gmail.com>]

    The following code is from execute method in SplitTransaction class
    If the region server shut down when the following code is executing.
    Some files may be left in HDFS forever?

    this.journal.add(JournalEntry.STARTED_REGION_A_CREATION);
    HRegion a = createDaughterRegion(this.hri_a, this.parent.rsServices);

    // Ditto
    this.journal.add(JournalEntry.STARTED_REGION_B_CREATION);
    HRegion b = createDaughterRegion(this.hri_b, this.parent.rsServices);

    // Edit parent in meta
    if (!testing) {
      MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
        this.parent.getRegionInfo(), a.getRegionInfo(), b.getRegionInfo());
    }

Re: Issues when splitting regions

[Stack <st...@duboce.net>]

On Tue, May 31, 2011 at 2:04 AM, jiangwen w <wj...@gmail.com> wrote:
>    The following code is from execute method in SplitTransaction class
>    If the region server shut down when the following code is executing.
>    Some files may be left in HDFS forever?
>

They could.

Check the CatalogJanitor and or the server shutdown handler.  Here we
will finish-off interrupted splits IFF at least the parent edit made
it up into .META.  There may be a hole here though where we should
recognize these abandoned daughter directories and do some cleanup.

IIRC, it has been suggested in the past that pre-split we write to the
WAL log what we are about to do.  If we crash in the middle of the
split, then on replay of edits when we trip over the special split
record, if we did not complete, we could clean up the abandoned
daughter regions written into HDFS?

But as Anty said, these abandoned daughters are more a nuisance than
harmful (We should clean them up though -- at a minimum hbck will flag
them).

St.Ack

Re: Issues when splitting regions

[Anty <an...@gmail.com>]

IMHO, yes.But it do no harm to HBase.

On Tue, May 31, 2011 at 5:04 PM, jiangwen w <wj...@gmail.com> wrote:

>    The following code is from execute method in SplitTransaction class
>    If the region server shut down when the following code is executing.
>    Some files may be left in HDFS forever?
>
>    this.journal.add(JournalEntry.STARTED_REGION_A_CREATION);
>    HRegion a = createDaughterRegion(this.hri_a, this.parent.rsServices);
>
>    // Ditto
>    this.journal.add(JournalEntry.STARTED_REGION_B_CREATION);
>    HRegion b = createDaughterRegion(this.hri_b, this.parent.rsServices);
>
>    // Edit parent in meta
>    if (!testing) {
>      MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
>        this.parent.getRegionInfo(), a.getRegionInfo(), b.getRegionInfo());
>    }
>



-- 
Best Regards
Anty Rao

Re: Issues when splitting regions

[Anty <an...@gmail.com>]

IMHO, yes.But it do no harm to HBase.

On Tue, May 31, 2011 at 5:04 PM, jiangwen w <wj...@gmail.com> wrote:

>    The following code is from execute method in SplitTransaction class
>    If the region server shut down when the following code is executing.
>    Some files may be left in HDFS forever?
>
>    this.journal.add(JournalEntry.STARTED_REGION_A_CREATION);
>    HRegion a = createDaughterRegion(this.hri_a, this.parent.rsServices);
>
>    // Ditto
>    this.journal.add(JournalEntry.STARTED_REGION_B_CREATION);
>    HRegion b = createDaughterRegion(this.hri_b, this.parent.rsServices);
>
>    // Edit parent in meta
>    if (!testing) {
>      MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
>        this.parent.getRegionInfo(), a.getRegionInfo(), b.getRegionInfo());
>    }
>



-- 
Best Regards
Anty Rao