You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Vitaliy Semochkin <vi...@gmail.com> on 2019/11/06 22:47:35 UTC
hbase connection internal pool size managment
Hi all,
I have many client fetching data from HBase
from a single jvm. When I reuse HBase connection the number of
requests per second that application can handle is relatively small,
however if I create a pool of connections and reuse it the number of
requests per second dramatically increases.
I suspect this is because HBase connection has a pool inside which is
relatively small. What parameter/parameters can be altered in order
to improve performance of single hbase connection? Are there any means
to increase maximum concurrent operations allowed?
Regards,
Vitaliy
Re: hbase connection internal pool size managment
Posted by yuhang li <89...@gmail.com>.
*hbase.client.ipc.pool.size* defines the number of conections(not threads)
to each region server,.
I'm not very sure if it would work, but I'm sure you should not reuse a
connection in a single jvm.
Vitaliy Semochkin <vi...@gmail.com> 于2019年11月7日周四 下午7:59写道：
> Thank you very much Yuhang Li!
>
> Do you think I also should increase
> hbase.client.ipc.pool.size
> if I have hundreds parallel threads executing Get?
>
> Regards,
> Vitaliy
>
> On Thu, Nov 7, 2019 at 1:06 PM yuhang li <89...@gmail.com> wrote:
> >
> > Yes,there is a thread pool in hbase connection.
> >
> > You can set the core pool size by *hbase.hconnection.threads.core*, and
> the
> > max pool size by *hbase.hconnection.threads.max*.
> >
> > Vitaliy Semochkin <vi...@gmail.com> 于2019年11月7日周四 上午6:48写道：
> >
> > > Hi all,
> > >
> > > I have many client fetching data from HBase
> > > from a single jvm. When I reuse HBase connection the number of
> > > requests per second that application can handle is relatively small,
> > > however if I create a pool of connections and reuse it the number of
> > > requests per second dramatically increases.
> > > I suspect this is because HBase connection has a pool inside which is
> > > relatively small. What parameter/parameters can be altered in order
> > > to improve performance of single hbase connection? Are there any means
> > > to increase maximum concurrent operations allowed?
> > >
> > > Regards,
> > > Vitaliy
> > >
>
Re: hbase connection internal pool size managment
Posted by Vitaliy Semochkin <vi...@gmail.com>.
Thank you very much Yuhang Li!
Do you think I also should increase
hbase.client.ipc.pool.size
if I have hundreds parallel threads executing Get?
Regards,
Vitaliy
On Thu, Nov 7, 2019 at 1:06 PM yuhang li <89...@gmail.com> wrote:
>
> Yes,there is a thread pool in hbase connection.
>
> You can set the core pool size by *hbase.hconnection.threads.core*, and the
> max pool size by *hbase.hconnection.threads.max*.
>
> Vitaliy Semochkin <vi...@gmail.com> 于2019年11月7日周四 上午6:48写道：
>
> > Hi all,
> >
> > I have many client fetching data from HBase
> > from a single jvm. When I reuse HBase connection the number of
> > requests per second that application can handle is relatively small,
> > however if I create a pool of connections and reuse it the number of
> > requests per second dramatically increases.
> > I suspect this is because HBase connection has a pool inside which is
> > relatively small. What parameter/parameters can be altered in order
> > to improve performance of single hbase connection? Are there any means
> > to increase maximum concurrent operations allowed?
> >
> > Regards,
> > Vitaliy
> >
Re: hbase connection internal pool size managment
Posted by yuhang li <89...@gmail.com>.
Yes,there is a thread pool in hbase connection.
You can set the core pool size by *hbase.hconnection.threads.core*, and the
max pool size by *hbase.hconnection.threads.max*.
Vitaliy Semochkin <vi...@gmail.com> 于2019年11月7日周四 上午6:48写道：
> Hi all,
>
> I have many client fetching data from HBase
> from a single jvm. When I reuse HBase connection the number of
> requests per second that application can handle is relatively small,
> however if I create a pool of connections and reuse it the number of
> requests per second dramatically increases.
> I suspect this is because HBase connection has a pool inside which is
> relatively small. What parameter/parameters can be altered in order
> to improve performance of single hbase connection? Are there any means
> to increase maximum concurrent operations allowed?
>
> Regards,
> Vitaliy
>