setCaching and setBatch

[John <jo...@gmail.com>]

Hi,

If I have a question about the setCaching function. As I know the caching
value influence how many rows will be cached in the memory of the region
server. So for example I have 10 rows with 20k columns and I set a scanner
with the caching option 5 and the batch option 1k. What will hapen If I
call the next() function?

As far as I know, the region server will cache 5 rows with 1k columns each
in the memory. If thats right, is there a way to cache the batches instead
of the rows? In this example I would like to cache 5 batches from row one
instead of 5 rows with each 1k columns.

kind regards

Re: setCaching and setBatch

[John <jo...@gmail.com>]

@Anoop: hmmm, I tried it some days ago and didn't see any improvement. But
maybe I'm wrong. Can anybody confirm this?

kind regards


2013/11/4 Asaf Mesika <as...@gmail.com>

> On this note:
> If I hammer a region server with scans that has let's say 100 rows caching
> - is there an OOME protections which lowers the caching in case ther server
> is stressed in memory? Or will it fail the region server with OOME?
>
> On Monday, November 4, 2013, Anoop John wrote:
>
> > Have u tested this?  AFAIK it happens the way u like.
> >
> > -Anoop-
> >
> > On Mon, Nov 4, 2013 at 10:49 PM, John <johnnyenglish739@gmail.com
> <javascript:;>>
> > wrote:
> >
> > > Hi,
> > >
> > > If I have a question about the setCaching function. As I know the
> caching
> > > value influence how many rows will be cached in the memory of the
> region
> > > server. So for example I have 10 rows with 20k columns and I set a
> > scanner
> > > with the caching option 5 and the batch option 1k. What will hapen If I
> > > call the next() function?
> > >
> > > As far as I know, the region server will cache 5 rows with 1k columns
> > each
> > > in the memory. If thats right, is there a way to cache the batches
> > instead
> > > of the rows? In this example I would like to cache 5 batches from row
> one
> > > instead of 5 rows with each 1k columns.
> > >
> > > kind regards
> > >
> >
>

RE: setCaching and setBatch

[Vladimir Rodionov <vr...@carrieriq.com>]

I think, setBatch is  your OOME protection as since it limits the total amount of KeyValues
processed in a single call to StoreScanner.next.

Best regards,
Vladimir Rodionov
Principal Platform Engineer
Carrier IQ, www.carrieriq.com
e-mail: vrodionov@carrieriq.com

________________________________________
From: Ted Yu [yuzhihong@gmail.com]
Sent: Monday, November 04, 2013 1:26 PM
To: user@hbase.apache.org
Subject: Re: setCaching and setBatch

bq. is there an OOME protections which lowers the caching in case ther
server is stressed in memory?

I am not aware of such mechanism.


On Mon, Nov 4, 2013 at 9:55 AM, Asaf Mesika <as...@gmail.com> wrote:

> On this note:
> If I hammer a region server with scans that has let's say 100 rows caching
> - is there an OOME protections which lowers the caching in case ther server
> is stressed in memory? Or will it fail the region server with OOME?
>
> On Monday, November 4, 2013, Anoop John wrote:
>
> > Have u tested this?  AFAIK it happens the way u like.
> >
> > -Anoop-
> >
> > On Mon, Nov 4, 2013 at 10:49 PM, John <johnnyenglish739@gmail.com
> <javascript:;>>
> > wrote:
> >
> > > Hi,
> > >
> > > If I have a question about the setCaching function. As I know the
> caching
> > > value influence how many rows will be cached in the memory of the
> region
> > > server. So for example I have 10 rows with 20k columns and I set a
> > scanner
> > > with the caching option 5 and the batch option 1k. What will hapen If I
> > > call the next() function?
> > >
> > > As far as I know, the region server will cache 5 rows with 1k columns
> > each
> > > in the memory. If thats right, is there a way to cache the batches
> > instead
> > > of the rows? In this example I would like to cache 5 batches from row
> one
> > > instead of 5 rows with each 1k columns.
> > >
> > > kind regards
> > >
> >
>

Confidentiality Notice:  The information contained in this message, including any attachments hereto, may be confidential and is intended to be read only by the individual or entity to whom this message is addressed. If the reader of this message is not the intended recipient or an agent or designee of the intended recipient, please note that any review, use, disclosure or distribution of this message or its attachments, in any form, is strictly prohibited.  If you have received this message in error, please immediately notify the sender and/or Notifications@carrieriq.com and delete or destroy any copy of this message and its attachments.

Re: setCaching and setBatch

[Ted Yu <yu...@gmail.com>]

bq. is there an OOME protections which lowers the caching in case ther
server is stressed in memory?

I am not aware of such mechanism.


On Mon, Nov 4, 2013 at 9:55 AM, Asaf Mesika <as...@gmail.com> wrote:

> On this note:
> If I hammer a region server with scans that has let's say 100 rows caching
> - is there an OOME protections which lowers the caching in case ther server
> is stressed in memory? Or will it fail the region server with OOME?
>
> On Monday, November 4, 2013, Anoop John wrote:
>
> > Have u tested this?  AFAIK it happens the way u like.
> >
> > -Anoop-
> >
> > On Mon, Nov 4, 2013 at 10:49 PM, John <johnnyenglish739@gmail.com
> <javascript:;>>
> > wrote:
> >
> > > Hi,
> > >
> > > If I have a question about the setCaching function. As I know the
> caching
> > > value influence how many rows will be cached in the memory of the
> region
> > > server. So for example I have 10 rows with 20k columns and I set a
> > scanner
> > > with the caching option 5 and the batch option 1k. What will hapen If I
> > > call the next() function?
> > >
> > > As far as I know, the region server will cache 5 rows with 1k columns
> > each
> > > in the memory. If thats right, is there a way to cache the batches
> > instead
> > > of the rows? In this example I would like to cache 5 batches from row
> one
> > > instead of 5 rows with each 1k columns.
> > >
> > > kind regards
> > >
> >
>

Re: setCaching and setBatch

[Asaf Mesika <as...@gmail.com>]

On this note:
If I hammer a region server with scans that has let's say 100 rows caching
- is there an OOME protections which lowers the caching in case ther server
is stressed in memory? Or will it fail the region server with OOME?

On Monday, November 4, 2013, Anoop John wrote:

> Have u tested this?  AFAIK it happens the way u like.
>
> -Anoop-
>
> On Mon, Nov 4, 2013 at 10:49 PM, John <johnnyenglish739@gmail.com<javascript:;>>
> wrote:
>
> > Hi,
> >
> > If I have a question about the setCaching function. As I know the caching
> > value influence how many rows will be cached in the memory of the region
> > server. So for example I have 10 rows with 20k columns and I set a
> scanner
> > with the caching option 5 and the batch option 1k. What will hapen If I
> > call the next() function?
> >
> > As far as I know, the region server will cache 5 rows with 1k columns
> each
> > in the memory. If thats right, is there a way to cache the batches
> instead
> > of the rows? In this example I would like to cache 5 batches from row one
> > instead of 5 rows with each 1k columns.
> >
> > kind regards
> >
>

Re: setCaching and setBatch

[Anoop John <an...@gmail.com>]

Have u tested this?  AFAIK it happens the way u like.

-Anoop-

On Mon, Nov 4, 2013 at 10:49 PM, John <jo...@gmail.com> wrote:

> Hi,
>
> If I have a question about the setCaching function. As I know the caching
> value influence how many rows will be cached in the memory of the region
> server. So for example I have 10 rows with 20k columns and I set a scanner
> with the caching option 5 and the batch option 1k. What will hapen If I
> call the next() function?
>
> As far as I know, the region server will cache 5 rows with 1k columns each
> in the memory. If thats right, is there a way to cache the batches instead
> of the rows? In this example I would like to cache 5 batches from row one
> instead of 5 rows with each 1k columns.
>
> kind regards
>