You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Amit Sela <am...@infolinks.com> on 2014/01/15 14:34:00 UTC
Is get a private case of scan ?
Hi all,
I was wondering if Get is implemented as a private case of scan ?
In HRegion, I see that the get passed is used to construct a Scan object
for the RegionScanner to use.
I was wondering if executing Scan(Get) vs Get from client api should have
any overhead ?
Thanks,
Amit.
Re: Is get a private case of scan ?
Posted by Amit Sela <am...@infolinks.com>.
I'm managing splitting manually and my algorithm ensures that.
On Jan 16, 2014 10:34 PM, "Ted Yu" <yu...@gmail.com> wrote:
> bq. be sure the second rowkey will be in the same region
>
> See 9.7.4.1 in http://hbase.apache.org/book.html#arch.region.splits
>
>
> On Thu, Jan 16, 2014 at 10:58 AM, Asaf Mesika <as...@gmail.com>
> wrote:
>
> > You also can't be sure the second rowkey will be in the same region which
> > is requires you to go on the network to another region server.
> >
> > On Thursday, January 16, 2014, Amit Sela <am...@infolinks.com> wrote:
> >
> > > I have a case where I want to "split" rows with a lot of qualifiers (a
> > very
> > > small amount of rows < 1%, with an exceptional number of qualifiers),
> > into
> > > a number of rows. Say like:
> > > row1.....
> > > row1_DELIMITER_UUID
> > > row1_DELIMITER_UUID2
> > > row2
> > > I was thinking of using a postGet() RegionObserver (the "split" rows
> will
> > > always be stored in the same region) but when digging into the code I
> > > noticed that Get is a Scan so I was wondering why not just scan for all
> > > row1s ?
> > >
> > >
> > > On Wed, Jan 15, 2014 at 10:53 PM, Stack <stack@duboce.net
> <javascript:;>>
> > > wrote:
> > >
> > > > On Wed, Jan 15, 2014 at 5:34 AM, Amit Sela <amits@infolinks.com
> > <javascript:;>>
> > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > I was wondering if Get is implemented as a private case of scan ?
> > > > > In HRegion, I see that the get passed is used to construct a Scan
> > > object
> > > > > for the RegionScanner to use.
> > > > >
> > > > >
> > > > A Get is a Scan, yes.
> > > >
> > > >
> > > > > I was wondering if executing Scan(Get) vs Get from client api
> should
> > > have
> > > > > any overhead ?
> > > > >
> > > >
> > > > There are differences in that Get will pread from HDFS where Scan
> will
> > > seek
> > > > and read under lock (and only fall back to pread when it notices
> > > contention
> > > > on the hfile).
> > > >
> > > > Why do you want to Scan(Get) instead of Get?
> > > >
> > > > St.Ack
> > > >
> > >
> >
>
Re: Is get a private case of scan ?
Posted by Ted Yu <yu...@gmail.com>.
bq. be sure the second rowkey will be in the same region
See 9.7.4.1 in http://hbase.apache.org/book.html#arch.region.splits
On Thu, Jan 16, 2014 at 10:58 AM, Asaf Mesika <as...@gmail.com> wrote:
> You also can't be sure the second rowkey will be in the same region which
> is requires you to go on the network to another region server.
>
> On Thursday, January 16, 2014, Amit Sela <am...@infolinks.com> wrote:
>
> > I have a case where I want to "split" rows with a lot of qualifiers (a
> very
> > small amount of rows < 1%, with an exceptional number of qualifiers),
> into
> > a number of rows. Say like:
> > row1.....
> > row1_DELIMITER_UUID
> > row1_DELIMITER_UUID2
> > row2
> > I was thinking of using a postGet() RegionObserver (the "split" rows will
> > always be stored in the same region) but when digging into the code I
> > noticed that Get is a Scan so I was wondering why not just scan for all
> > row1s ?
> >
> >
> > On Wed, Jan 15, 2014 at 10:53 PM, Stack <stack@duboce.net<javascript:;>>
> > wrote:
> >
> > > On Wed, Jan 15, 2014 at 5:34 AM, Amit Sela <amits@infolinks.com
> <javascript:;>>
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I was wondering if Get is implemented as a private case of scan ?
> > > > In HRegion, I see that the get passed is used to construct a Scan
> > object
> > > > for the RegionScanner to use.
> > > >
> > > >
> > > A Get is a Scan, yes.
> > >
> > >
> > > > I was wondering if executing Scan(Get) vs Get from client api should
> > have
> > > > any overhead ?
> > > >
> > >
> > > There are differences in that Get will pread from HDFS where Scan will
> > seek
> > > and read under lock (and only fall back to pread when it notices
> > contention
> > > on the hfile).
> > >
> > > Why do you want to Scan(Get) instead of Get?
> > >
> > > St.Ack
> > >
> >
>
Re: Is get a private case of scan ?
Posted by Asaf Mesika <as...@gmail.com>.
You also can't be sure the second rowkey will be in the same region which
is requires you to go on the network to another region server.
On Thursday, January 16, 2014, Amit Sela <am...@infolinks.com> wrote:
> I have a case where I want to "split" rows with a lot of qualifiers (a very
> small amount of rows < 1%, with an exceptional number of qualifiers), into
> a number of rows. Say like:
> row1.....
> row1_DELIMITER_UUID
> row1_DELIMITER_UUID2
> row2
> I was thinking of using a postGet() RegionObserver (the "split" rows will
> always be stored in the same region) but when digging into the code I
> noticed that Get is a Scan so I was wondering why not just scan for all
> row1s ?
>
>
> On Wed, Jan 15, 2014 at 10:53 PM, Stack <stack@duboce.net <javascript:;>>
> wrote:
>
> > On Wed, Jan 15, 2014 at 5:34 AM, Amit Sela <amits@infolinks.com<javascript:;>>
> wrote:
> >
> > > Hi all,
> > >
> > > I was wondering if Get is implemented as a private case of scan ?
> > > In HRegion, I see that the get passed is used to construct a Scan
> object
> > > for the RegionScanner to use.
> > >
> > >
> > A Get is a Scan, yes.
> >
> >
> > > I was wondering if executing Scan(Get) vs Get from client api should
> have
> > > any overhead ?
> > >
> >
> > There are differences in that Get will pread from HDFS where Scan will
> seek
> > and read under lock (and only fall back to pread when it notices
> contention
> > on the hfile).
> >
> > Why do you want to Scan(Get) instead of Get?
> >
> > St.Ack
> >
>
Re: Is get a private case of scan ?
Posted by Amit Sela <am...@infolinks.com>.
I have a case where I want to "split" rows with a lot of qualifiers (a very
small amount of rows < 1%, with an exceptional number of qualifiers), into
a number of rows. Say like:
row1.....
row1_DELIMITER_UUID
row1_DELIMITER_UUID2
row2
I was thinking of using a postGet() RegionObserver (the "split" rows will
always be stored in the same region) but when digging into the code I
noticed that Get is a Scan so I was wondering why not just scan for all
row1s ?
On Wed, Jan 15, 2014 at 10:53 PM, Stack <st...@duboce.net> wrote:
> On Wed, Jan 15, 2014 at 5:34 AM, Amit Sela <am...@infolinks.com> wrote:
>
> > Hi all,
> >
> > I was wondering if Get is implemented as a private case of scan ?
> > In HRegion, I see that the get passed is used to construct a Scan object
> > for the RegionScanner to use.
> >
> >
> A Get is a Scan, yes.
>
>
> > I was wondering if executing Scan(Get) vs Get from client api should have
> > any overhead ?
> >
>
> There are differences in that Get will pread from HDFS where Scan will seek
> and read under lock (and only fall back to pread when it notices contention
> on the hfile).
>
> Why do you want to Scan(Get) instead of Get?
>
> St.Ack
>
Re: Is get a private case of scan ?
Posted by Stack <st...@duboce.net>.
On Wed, Jan 15, 2014 at 5:34 AM, Amit Sela <am...@infolinks.com> wrote:
> Hi all,
>
> I was wondering if Get is implemented as a private case of scan ?
> In HRegion, I see that the get passed is used to construct a Scan object
> for the RegionScanner to use.
>
>
A Get is a Scan, yes.
> I was wondering if executing Scan(Get) vs Get from client api should have
> any overhead ?
>
There are differences in that Get will pread from HDFS where Scan will seek
and read under lock (and only fall back to pread when it notices contention
on the hfile).
Why do you want to Scan(Get) instead of Get?
St.Ack