You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Vladimir Rodionov <vl...@gmail.com> on 2015/02/25 19:39:38 UTC
Snapshot internals questions
Hi, folks
1. In EnabledTableSnapshotHandler in
protected void snapshotRegions(List<Pair<HRegionInfo, ServerName>> regions)
why do we make snapshot for offline and split regions after all online
regions are done?
2. Do we have a possible race condition in SnapshotManifest:
a. When we get a list of a store files in
public void addRegion(final HRegion region)
b. then we iterate over this list and do some work
c. then we close snapshot for region and save manifest to a file
between a. and c. some files from list in a. can become invalid (deleted by
HFileCleaner).
I checked the code, no exception is raised if file is no longer exists
during this phase.
thanks,
-Vlad
Re: Snapshot internals questions
Posted by Jerry He <je...@gmail.com>.
Almost forget that!
Thanks!
Re: Snapshot internals questions
Posted by Vladimir Rodionov <vl...@gmail.com>.
Yes, thanks
found already.
-Vlad
On Wed, Feb 25, 2015 at 4:04 PM, Matteo Bertozzi <th...@gmail.com>
wrote:
> On Wed, Feb 25, 2015 at 6:39 PM, Vladimir Rodionov <vladrodionov@gmail.com
> >
> wrote:
>
> > 2. Do we have a possible race condition in SnapshotManifest:
> > a. When we get a list of a store files in
> > public void addRegion(final HRegion region)
> > b. then we iterate over this list and do some work
> > c. then we close snapshot for region and save manifest to a file
> >
> > between a. and c. some files from list in a. can become invalid (deleted
> by
> > HFileCleaner).
> >
> > I checked the code, no exception is raised if file is no longer exists
> > during this phase.
> >
>
> there is a sanity check on the master before marking the snapshot as
> successful. MasterSnapshotVerifier
>
Re: Snapshot internals questions
Posted by Matteo Bertozzi <th...@gmail.com>.
On Wed, Feb 25, 2015 at 6:39 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:
> 2. Do we have a possible race condition in SnapshotManifest:
> a. When we get a list of a store files in
> public void addRegion(final HRegion region)
> b. then we iterate over this list and do some work
> c. then we close snapshot for region and save manifest to a file
>
> between a. and c. some files from list in a. can become invalid (deleted by
> HFileCleaner).
>
> I checked the code, no exception is raised if file is no longer exists
> during this phase.
>
there is a sanity check on the master before marking the snapshot as
successful. MasterSnapshotVerifier
Re: Snapshot internals questions
Posted by Vladimir Rodionov <vl...@gmail.com>.
Jerry,
merge and compaction bypass region lock, just check the code .
in startRegionOperation:
if (op == Operation.MERGE_REGION || op == Operation.SPLIT_REGION
|| op == Operation.COMPACT_REGION) {
// split, merge or compact region doesn't need to check the
closing/closed state or lock the
// region
return;
}
-Vlad
On Wed, Feb 25, 2015 at 1:47 PM, Jerry He <je...@gmail.com> wrote:
> Hi, Vlad
>
> For question 1. see HBASE-8760
> <https://issues.apache.org/jira/browse/HBASE-8760>.
> For question 2, the region is under startRegionOperation() --> read lock.
>
> Thanks,
>
> Jerry
>
> On Wed, Feb 25, 2015 at 10:39 AM, Vladimir Rodionov <
> vladrodionov@gmail.com>
> wrote:
>
> > Hi, folks
> >
> > 1. In EnabledTableSnapshotHandler in
> >
> > protected void snapshotRegions(List<Pair<HRegionInfo, ServerName>>
> regions)
> >
> > why do we make snapshot for offline and split regions after all online
> > regions are done?
> >
> >
> > 2. Do we have a possible race condition in SnapshotManifest:
> >
> > a. When we get a list of a store files in
> >
> > public void addRegion(final HRegion region)
> >
> > b. then we iterate over this list and do some work
> >
> > c. then we close snapshot for region and save manifest to a file
> >
> > between a. and c. some files from list in a. can become invalid (deleted
> by
> > HFileCleaner).
> >
> > I checked the code, no exception is raised if file is no longer exists
> > during this phase.
> >
> > thanks,
> > -Vlad
> >
>
Re: Snapshot internals questions
Posted by Jerry He <je...@gmail.com>.
Hi, Vlad
For question 1. see HBASE-8760
<https://issues.apache.org/jira/browse/HBASE-8760>.
For question 2, the region is under startRegionOperation() --> read lock.
Thanks,
Jerry
On Wed, Feb 25, 2015 at 10:39 AM, Vladimir Rodionov <vl...@gmail.com>
wrote:
> Hi, folks
>
> 1. In EnabledTableSnapshotHandler in
>
> protected void snapshotRegions(List<Pair<HRegionInfo, ServerName>> regions)
>
> why do we make snapshot for offline and split regions after all online
> regions are done?
>
>
> 2. Do we have a possible race condition in SnapshotManifest:
>
> a. When we get a list of a store files in
>
> public void addRegion(final HRegion region)
>
> b. then we iterate over this list and do some work
>
> c. then we close snapshot for region and save manifest to a file
>
> between a. and c. some files from list in a. can become invalid (deleted by
> HFileCleaner).
>
> I checked the code, no exception is raised if file is no longer exists
> during this phase.
>
> thanks,
> -Vlad
>