Increased response time of hbase calls

[Deepak Khandelwal <dk...@gmail.com>]

Hi all

I am facing an issue while accessing data from an hbase m7 table which has
about 50 million records.

In a single Api request, we make 3 calls to hbase m7.
1. Single Multi get to fetch about 30 records
2. Single multi-put to update about 500 records
3. Single multi-get to fetch about 15 records

We consistently get the response in less than 200 seconds for approx
99%calls. We have a tps of about 200 with 8vm's.
But we get issue everyday between 4pm and 6pm when Api response time gets
significant increase to from 200ms to 7-8sec. This happens because we have
a daily batch load That runs between 4and 6pm that puts multiple entries
into same hbase table.

We are trying to find a solution to this problem that why response time
increases when batch load runs. We cannot change the time of batch job. Is
there anything we could do to resolve this issue?any help or pointers would
be much appreciated. Thanks

Re: Increased response time of hbase calls

[Enis Söztutar <en...@apache.org>]

Please also note that m7 IS NOT HBase and have no connection with Apache
HBase at all. Please do not let your vendor tell otherwise.

Enis

On Thu, Sep 22, 2016 at 12:00 AM, Deepak Khandelwal <
dkhandelwal.tia@gmail.com> wrote:

> Ok thx everyone.
>
> Will check with Mapr
>
> On Wednesday, September 21, 2016, Heng Chen <he...@gmail.com>
> wrote:
>
> > Not sure hbase m7 is which version of hbase in community.
> >
> > Is your batch load job some kind of bulk load or just call HTable API
> > to dump data to HBase?
> >
> >
> > 2016-09-22 14:30 GMT+08:00 Dima Spivak <dimaspivak@apache.org
> > <javascript:;>>:
> > > Hey Deepak,
> > >
> > > Assuming I understand your question, I think you'd be better served
> > > reaching out to MapR directly. Our community isn't involved in M7 so
> the
> > > average user (or dev) wouldn't know about the ins and outs of that
> > > offering.
> > >
> > > On Wednesday, September 21, 2016, Deepak Khandelwal <
> > > dkhandelwal.tia@gmail.com <javascript:;>> wrote:
> > >
> > >> Hi all
> > >>
> > >> I am facing an issue while accessing data from an hbase m7 table which
> > has
> > >> about 50 million records.
> > >>
> > >> In a single Api request, we make 3 calls to hbase m7.
> > >> 1. Single Multi get to fetch about 30 records
> > >> 2. Single multi-put to update about 500 records
> > >> 3. Single multi-get to fetch about 15 records
> > >>
> > >> We consistently get the response in less than 200 seconds for approx
> > >> 99%calls. We have a tps of about 200 with 8vm's.
> > >> But we get issue everyday between 4pm and 6pm when Api response time
> > gets
> > >> significant increase to from 200ms to 7-8sec. This happens because we
> > have
> > >> a daily batch load That runs between 4and 6pm that puts multiple
> entries
> > >> into same hbase table.
> > >>
> > >> We are trying to find a solution to this problem that why response
> time
> > >> increases when batch load runs. We cannot change the time of batch
> job.
> > Is
> > >> there anything we could do to resolve this issue?any help or pointers
> > would
> > >> be much appreciated. Thanks
> > >>
> > >
> > >
> > > --
> > > -Dima
> >
>

Re: Increased response time of hbase calls

[Deepak Khandelwal <dk...@gmail.com>]

Ok thx everyone.

Will check with Mapr

On Wednesday, September 21, 2016, Heng Chen <he...@gmail.com>
wrote:

> Not sure hbase m7 is which version of hbase in community.
>
> Is your batch load job some kind of bulk load or just call HTable API
> to dump data to HBase?
>
>
> 2016-09-22 14:30 GMT+08:00 Dima Spivak <dimaspivak@apache.org
> <javascript:;>>:
> > Hey Deepak,
> >
> > Assuming I understand your question, I think you'd be better served
> > reaching out to MapR directly. Our community isn't involved in M7 so the
> > average user (or dev) wouldn't know about the ins and outs of that
> > offering.
> >
> > On Wednesday, September 21, 2016, Deepak Khandelwal <
> > dkhandelwal.tia@gmail.com <javascript:;>> wrote:
> >
> >> Hi all
> >>
> >> I am facing an issue while accessing data from an hbase m7 table which
> has
> >> about 50 million records.
> >>
> >> In a single Api request, we make 3 calls to hbase m7.
> >> 1. Single Multi get to fetch about 30 records
> >> 2. Single multi-put to update about 500 records
> >> 3. Single multi-get to fetch about 15 records
> >>
> >> We consistently get the response in less than 200 seconds for approx
> >> 99%calls. We have a tps of about 200 with 8vm's.
> >> But we get issue everyday between 4pm and 6pm when Api response time
> gets
> >> significant increase to from 200ms to 7-8sec. This happens because we
> have
> >> a daily batch load That runs between 4and 6pm that puts multiple entries
> >> into same hbase table.
> >>
> >> We are trying to find a solution to this problem that why response time
> >> increases when batch load runs. We cannot change the time of batch job.
> Is
> >> there anything we could do to resolve this issue?any help or pointers
> would
> >> be much appreciated. Thanks
> >>
> >
> >
> > --
> > -Dima
>

Re: Increased response time of hbase calls

[Heng Chen <he...@gmail.com>]

Not sure hbase m7 is which version of hbase in community.

Is your batch load job some kind of bulk load or just call HTable API
to dump data to HBase?


2016-09-22 14:30 GMT+08:00 Dima Spivak <di...@apache.org>:
> Hey Deepak,
>
> Assuming I understand your question, I think you'd be better served
> reaching out to MapR directly. Our community isn't involved in M7 so the
> average user (or dev) wouldn't know about the ins and outs of that
> offering.
>
> On Wednesday, September 21, 2016, Deepak Khandelwal <
> dkhandelwal.tia@gmail.com> wrote:
>
>> Hi all
>>
>> I am facing an issue while accessing data from an hbase m7 table which has
>> about 50 million records.
>>
>> In a single Api request, we make 3 calls to hbase m7.
>> 1. Single Multi get to fetch about 30 records
>> 2. Single multi-put to update about 500 records
>> 3. Single multi-get to fetch about 15 records
>>
>> We consistently get the response in less than 200 seconds for approx
>> 99%calls. We have a tps of about 200 with 8vm's.
>> But we get issue everyday between 4pm and 6pm when Api response time gets
>> significant increase to from 200ms to 7-8sec. This happens because we have
>> a daily batch load That runs between 4and 6pm that puts multiple entries
>> into same hbase table.
>>
>> We are trying to find a solution to this problem that why response time
>> increases when batch load runs. We cannot change the time of batch job. Is
>> there anything we could do to resolve this issue?any help or pointers would
>> be much appreciated. Thanks
>>
>
>
> --
> -Dima

Re: Increased response time of hbase calls

[Dima Spivak <di...@apache.org>]

Hey Deepak,

Assuming I understand your question, I think you'd be better served
reaching out to MapR directly. Our community isn't involved in M7 so the
average user (or dev) wouldn't know about the ins and outs of that
offering.

On Wednesday, September 21, 2016, Deepak Khandelwal <
dkhandelwal.tia@gmail.com> wrote:

> Hi all
>
> I am facing an issue while accessing data from an hbase m7 table which has
> about 50 million records.
>
> In a single Api request, we make 3 calls to hbase m7.
> 1. Single Multi get to fetch about 30 records
> 2. Single multi-put to update about 500 records
> 3. Single multi-get to fetch about 15 records
>
> We consistently get the response in less than 200 seconds for approx
> 99%calls. We have a tps of about 200 with 8vm's.
> But we get issue everyday between 4pm and 6pm when Api response time gets
> significant increase to from 200ms to 7-8sec. This happens because we have
> a daily batch load That runs between 4and 6pm that puts multiple entries
> into same hbase table.
>
> We are trying to find a solution to this problem that why response time
> increases when batch load runs. We cannot change the time of batch job. Is
> there anything we could do to resolve this issue?any help or pointers would
> be much appreciated. Thanks
>


-- 
-Dima

Re: Increased response time of hbase calls

[Vladimir Rodionov <vl...@gmail.com>]

m7 is vendor - specific version of HBase (MapR)
You better ask this question on MapR user list.

-Vlad

On Wed, Sep 21, 2016 at 10:28 PM, Ted Yu <yu...@gmail.com> wrote:

> Which hbase release are you using ?
>
> Can you tell us the values for handler related config such as
> hbase.regionserver.handler.count ?
>
> How many regions does m7 have ?
>
> Have many servers does m7 span ? Are regions of m7 spread evenly ?
>
> Thanks
> ​
>

Re: Increased response time of hbase calls

[Ted Yu <yu...@gmail.com>]

Which hbase release are you using ?

Can you tell us the values for handler related config such as
hbase.regionserver.handler.count ?

How many regions does m7 have ?

Have many servers does m7 span ? Are regions of m7 spread evenly ?

Thanks
​