Query time shoots up because of minor compaction.

[munishmhr <mu...@gmail.com>]

Our application does continuous bulkload through out the day and at the same
time we are also doing query. From the logs i can see minor compaction
happening timings of which range from ~2sec to ~2mints at the same time
query hbase response time shoots up ranging ~2sec to ~27sec.

Our application is using output format 
https://issues.apache.org/jira/browse/HBASE-18161
<https://issues.apache.org/jira/browse/HBASE-18161>  

Now my question is if minor compaction should really cause this response
time if so what changes can be done to minimize the latency.

One solution that i can come up with is to use puts in reducer it self by
writing some custom output format.
By this data will be always in memstore and minor compaction can be stopped
for every bulkload.



--
Sent from: http://apache-hbase.679495.n3.nabble.com/HBase-User-f4020416.html

Re: Query time shoots up because of minor compaction.

[munishmhr <mu...@gmail.com>]

Yes i was trying understand if minor compaction can really effect response
time.

Other than that i don't see any other activity on our servers.

about 18161, I am working with guy who wrote it so we are using that output
format.



--
Sent from: http://apache-hbase.679495.n3.nabble.com/HBase-User-f4020416.html

Re: Query time shoots up because of minor compaction.

[Ted Yu <yu...@gmail.com>]

bq. range from ~2sec to ~2mints at the same time query hbase response time
shoots up

By the above I assume you have established correlation between the timing
of minor compaction and slow response.
Was there other activity from region server log which could affect response
time ?

BTW HBASE-18161 <https://issues.apache.org/jira/browse/HBASE-18161> was
only integrated to branch-2 which has no release yet.

Which hbase release did the backport go to ?

Cheers

On Sun, Nov 19, 2017 at 6:55 AM, munishmhr <mu...@gmail.com> wrote:

> Our application does continuous bulkload through out the day and at the
> same
> time we are also doing query. From the logs i can see minor compaction
> happening timings of which range from ~2sec to ~2mints at the same time
> query hbase response time shoots up ranging ~2sec to ~27sec.
>
> Our application is using output format
> https://issues.apache.org/jira/browse/HBASE-18161
> <https://issues.apache.org/jira/browse/HBASE-18161>
>
> Now my question is if minor compaction should really cause this response
> time if so what changes can be done to minimize the latency.
>
> One solution that i can come up with is to use puts in reducer it self by
> writing some custom output format.
> By this data will be always in memstore and minor compaction can be stopped
> for every bulkload.
>
>
>
> --
> Sent from: http://apache-hbase.679495.n3.nabble.com/HBase-User-
> f4020416.html
>