You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by sleimanjneidi <jn...@gmail.com> on 2015/02/03 15:47:29 UTC
managing HConnection
Hi all,
I am using hbase-0.98.1-cdh5.1.4 client and I am a bit confused by the documentation of HConnection. The document says the following:
HConnection instances can be shared. Sharing is usually what you want because rather than each HConnection instance having to do its own discovery of regions out on the cluster, instead, all clients get to share the one cache of locations. HConnectionManager does the sharing for you if you go by it getting connections. Sharing makes cleanup of HConnections awkward. .
So now I have a simple question: Can I share the same HConnection instance in my entire application?
And write some magic code to know when to close or never close at all?
Or I have to create an instance and close it every time I do a CRUD operation ?
Many thanks
Re: managing HConnection
Posted by Serega Sheypak <se...@gmail.com>.
Hi, I'm closing it in servlet.destroy. I didn't see any problems here for
months. I'm using servlet lifecycle to deal with hconnection.
вторник, 17 февраля 2015 г. пользователь Liu, Ming (HPIT-GADSC) написал:
> Hi,
>
> Thank you Serega for the helpful reply and thanks Jneidi for asking this.
> I have similar confusion.
> So Serega, when does your application finally close the HConnection? Or
> the connection is NEVER closed as long as your application is running? Is
> it OK to NOT close the HConnection and the application exit directly?
> My application is a long-running service, accept user request and do CRUD
> to hbase. So I would like to use your model here. But, is it reasonable to
> keep that HConnection open a very long time, for example months? Is there
> any potential problem I need to take care?
> Also as David Chen asked, if all threads share same HConnection, it may
> has limitation to support high throughput, so a pool of Connections maybe
> better?
>
> Thanks,
> Ming
>
> -----Original Message-----
> From: Serega Sheypak [mailto:serega.sheypak@gmail.com <javascript:;>]
> Sent: Wednesday, February 04, 2015 1:02 AM
> To: user
> Subject: Re: managing HConnection
>
> Hi, guys from group helped me a lot. I did solve pretty the same problem
> (CRUD web-app)
>
> 1. Use single instance of HConnection per application.
> 2. Instantiate it once.
> 3. create HTable instance for each CRUD operation and safely close it
> (try-catch-finally). Use the same HConnection to create any HTable for CRUD
> operation.
> 4. DO NOT close HConnection after CRUD operation
>
> I have logic controllers which get HConnection injection in
> HttpServlet.init method.
> So I have 5 HConnection instances per application created during servlet
> initialization
>
>
> 2015-02-03 18:12 GMT+03:00 Ted Yu <yuzhihong@gmail.com <javascript:;>>:
>
> > Please see '61.1. Cluster Connections' under
> > http://hbase.apache.org/book.html#architecture.client
> >
> > Cheers
> >
> > On Tue, Feb 3, 2015 at 6:47 AM, sleimanjneidi
> > <jneidi.sleiman@gmail.com <javascript:;>>
> > wrote:
> >
> > > Hi all,
> > > I am using hbase-0.98.1-cdh5.1.4 client and I am a bit confused by
> > > the documentation of HConnection. The document says the following:
> > >
> > > HConnection instances can be shared. Sharing is usually what you
> > > want because rather than each HConnection instance having to do its
> > > own discovery of regions out on the cluster, instead, all clients
> > > get to
> > share
> > > the one cache of locations. HConnectionManager does the sharing for
> > > you
> > if
> > > you go by it getting connections. Sharing makes cleanup of
> > > HConnections awkward. .
> > >
> > > So now I have a simple question: Can I share the same HConnection
> > instance
> > > in my entire application?
> > > And write some magic code to know when to close or never close at all?
> > > Or I have to create an instance and close it every time I do a CRUD
> > > operation ?
> > >
> > > Many thanks
> > >
> > >
> > >
> >
>
RE: managing HConnection
Posted by "Liu, Ming (HPIT-GADSC)" <mi...@hp.com>.
Hi,
Thank you Serega for the helpful reply and thanks Jneidi for asking this. I have similar confusion.
So Serega, when does your application finally close the HConnection? Or the connection is NEVER closed as long as your application is running? Is it OK to NOT close the HConnection and the application exit directly?
My application is a long-running service, accept user request and do CRUD to hbase. So I would like to use your model here. But, is it reasonable to keep that HConnection open a very long time, for example months? Is there any potential problem I need to take care?
Also as David Chen asked, if all threads share same HConnection, it may has limitation to support high throughput, so a pool of Connections maybe better?
Thanks,
Ming
-----Original Message-----
From: Serega Sheypak [mailto:serega.sheypak@gmail.com]
Sent: Wednesday, February 04, 2015 1:02 AM
To: user
Subject: Re: managing HConnection
Hi, guys from group helped me a lot. I did solve pretty the same problem (CRUD web-app)
1. Use single instance of HConnection per application.
2. Instantiate it once.
3. create HTable instance for each CRUD operation and safely close it (try-catch-finally). Use the same HConnection to create any HTable for CRUD operation.
4. DO NOT close HConnection after CRUD operation
I have logic controllers which get HConnection injection in HttpServlet.init method.
So I have 5 HConnection instances per application created during servlet initialization
2015-02-03 18:12 GMT+03:00 Ted Yu <yu...@gmail.com>:
> Please see '61.1. Cluster Connections' under
> http://hbase.apache.org/book.html#architecture.client
>
> Cheers
>
> On Tue, Feb 3, 2015 at 6:47 AM, sleimanjneidi
> <jn...@gmail.com>
> wrote:
>
> > Hi all,
> > I am using hbase-0.98.1-cdh5.1.4 client and I am a bit confused by
> > the documentation of HConnection. The document says the following:
> >
> > HConnection instances can be shared. Sharing is usually what you
> > want because rather than each HConnection instance having to do its
> > own discovery of regions out on the cluster, instead, all clients
> > get to
> share
> > the one cache of locations. HConnectionManager does the sharing for
> > you
> if
> > you go by it getting connections. Sharing makes cleanup of
> > HConnections awkward. .
> >
> > So now I have a simple question: Can I share the same HConnection
> instance
> > in my entire application?
> > And write some magic code to know when to close or never close at all?
> > Or I have to create an instance and close it every time I do a CRUD
> > operation ?
> >
> > Many thanks
> >
> >
> >
>
Re: Re: Re: managing HConnection
Posted by Sleiman Jneidi <jn...@gmail.com>.
That's true, we can share the connection, but the problem is, HConnection
implements Closeable, and hence its close method is public. If you share
the same connection in your api, you always have to check if the current
connection is closed before you return it to the client, check previous
emails to see how I was trying to achieve that.
Now, I am thinking of creating a wrapper around HConnection that overrides
HConnection close method and make it empty, just to make sure it's never
closed
On Sun, Feb 15, 2015 at 8:26 PM, Serega Sheypak <se...@gmail.com>
wrote:
> I don't understand you.
> There is a single instance of servlet per application.
> Servlet.init method called once. Here you can instantiate HConnection and
> solve ANY concurrency problems. HConnection is tread-safe. Just don't close
> it and reuse.
> Then just use HConnection to get HTable.
>
> What problem you are trying to solve?
>
>
> 2015-02-15 9:23 GMT+03:00 David chen <c7...@163.com>:
>
> > If sharing one HConnection for the whole application, when concurrency to
> > access servlets increases to a threshold, whether or not it will
> influence
> > the application performance? But if increasing the sharing HConnection
> > number, the problem will be relieved?
>
Re:Re: Re: managing HConnection
Posted by David chen <c7...@163.com>.
In your scene, per servlet used a HConnecton object, so the response for 50000 rpm is so easy.
If these servlets should share the only one HConnecton object, whether or not the response latency for 50000 rpm would increase?
Re: Re: managing HConnection
Posted by Serega Sheypak <se...@gmail.com>.
Newrelic shows 50K RPM
each request to servlet == 1-3 put/get to HBase. I have mixed workload.
Is it strange :) ?
2015-02-16 10:37 GMT+03:00 David chen <c7...@163.com>:
> 50000 rpm? I am curious how the result is concluded?
Re:Re: managing HConnection
Posted by David chen <c7...@163.com>.
50000 rpm? I am curious how the result is concluded?
Re: managing HConnection
Posted by Serega Sheypak <se...@gmail.com>.
It can. 50000 rpm, no problem.
понедельник, 16 февраля 2015 г. пользователь David chen написал:
> Sorry for the unclear represent.
> My problem is that whether or not a sharing Honnection can bear too many
> query requests?
Re:Re: Re: Re: managing HConnection
Posted by David chen <c7...@163.com>.
Sorry for the unclear represent.
My problem is that whether or not a sharing Honnection can bear too many query requests?
Re: Re: Re: managing HConnection
Posted by Serega Sheypak <se...@gmail.com>.
I don't understand you.
There is a single instance of servlet per application.
Servlet.init method called once. Here you can instantiate HConnection and
solve ANY concurrency problems. HConnection is tread-safe. Just don't close
it and reuse.
Then just use HConnection to get HTable.
What problem you are trying to solve?
2015-02-15 9:23 GMT+03:00 David chen <c7...@163.com>:
> If sharing one HConnection for the whole application, when concurrency to
> access servlets increases to a threshold, whether or not it will influence
> the application performance? But if increasing the sharing HConnection
> number, the problem will be relieved?
Re:Re: Re: managing HConnection
Posted by David chen <c7...@163.com>.
If sharing one HConnection for the whole application, when concurrency to access servlets increases to a threshold, whether or not it will influence the application performance? But if increasing the sharing HConnection number, the problem will be relieved?
Re: Re: managing HConnection
Posted by Serega Sheypak <se...@gmail.com>.
What's the problem to call HConnectionManager.getConnection in Servlet.init
method and pass it to your class responsible for HBase interaction?
2015-02-13 14:49 GMT+03:00 Sleiman Jneidi <jn...@gmail.com>:
> a single HConnection
>
> On Fri, Feb 13, 2015 at 11:12 AM, Serega Sheypak <serega.sheypak@gmail.com
> >
> wrote:
>
> > What are you trying to achieve?
> >
> > 2015-02-13 12:36 GMT+03:00 Sleiman Jneidi <jn...@gmail.com>:
> >
> > > To be honest guys I am still confused, especially that that HConnection
> > > implements Closeable and hence everyone has the right to close the
> > > connection. I wrote this code to manage connections but I am not sure
> > about
> > > its correctness.
> > >
> > >
> > > private static class HConnectionProvider {
> > >
> > > private static HConnection hConnection;
> > >
> > > private static final Lock LOCK = new ReentrantLock();
> > >
> > > static {
> > >
> > > hConnection = createNewConnection();
> > >
> > > Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
> > >
> > > @Override
> > >
> > > public void run() {
> > >
> > > if(hConnection!=null && !hConnection.isClosed()){
> > >
> > > try {
> > >
> > > hConnection.close();
> > >
> > > } catch (IOException e) {
> > >
> > > e.printStackTrace();
> > >
> > > }
> > >
> > > }
> > >
> > > }
> > >
> > > }));
> > >
> > > }
> > >
> > > public static HConnection connection(){
> > >
> > > if(!hConnection.isClosed()){
> > >
> > > return hConnection;
> > >
> > > }
> > >
> > > boolean acquired = false;
> > >
> > > try{
> > >
> > > acquired = LOCK.tryLock(5,TimeUnit.SECONDS);
> > >
> > > if(hConnection.isClosed()){
> > >
> > > hConnection = createNewConnection();
> > >
> > > }
> > >
> > > return hConnection;
> > >
> > > } catch (InterruptedException e) {
> > >
> > > throw new RuntimeException(e);
> > >
> > > }finally{
> > >
> > > if(acquired){
> > >
> > > LOCK.unlock();
> > >
> > > }
> > >
> > > }
> > >
> > > }
> > >
> > > private static HConnection createNewConnection(){
> > >
> > > try {
> > >
> > > HConnection connection = HConnectionManager.createConnection(config);
> > >
> > > return connection;
> > >
> > > } catch (IOException e) {
> > >
> > > throw new RuntimeException(e);
> > >
> > > }
> > >
> > > }
> > >
> > > }
> > >
> > > On Fri, Feb 13, 2015 at 8:57 AM, Serega Sheypak <
> > serega.sheypak@gmail.com>
> > > wrote:
> > >
> > > > Hi, really, I can share one Hconnection for the whole application.
> > > > It's done by design. I have several servlets. Each servlet has 1-2
> > > > controllers working with hbase internally (put/get/e.t.c)
> > > > Right now I don't see any reason to refactor code and share single
> > > > HConnection for all controllers in servlets.
> > > >
> > > >
> > > > 2015-02-13 6:56 GMT+03:00 David chen <c7...@163.com>:
> > > >
> > > > > Hi Serega,
> > > > > I am very interesting in the reason why per application need to
> > create
> > > 5
> > > > > instead of only one HConnection instances during servlet
> > > initialization?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > At 2015-02-04 01:01:38, "Serega Sheypak" <serega.sheypak@gmail.com
> >
> > > > wrote:
> > > > > >Hi, guys from group helped me a lot. I did solve pretty the same
> > > problem
> > > > > >(CRUD web-app)
> > > > > >
> > > > > >1. Use single instance of HConnection per application.
> > > > > >2. Instantiate it once.
> > > > > >3. create HTable instance for each CRUD operation and safely close
> > it
> > > > > >(try-catch-finally). Use the same HConnection to create any HTable
> > for
> > > > > CRUD
> > > > > >operation.
> > > > > >4. DO NOT close HConnection after CRUD operation
> > > > > >
> > > > > >I have logic controllers which get HConnection injection in
> > > > > >HttpServlet.init method.
> > > > > >So I have 5 HConnection instances per application created during
> > > servlet
> > > > > >initialization
> > > > > >
> > > > > >
> > > > > >2015-02-03 18:12 GMT+03:00 Ted Yu <yu...@gmail.com>:
> > > > > >
> > > > > >> Please see '61.1. Cluster Connections' under
> > > > > >> http://hbase.apache.org/book.html#architecture.client
> > > > > >>
> > > > > >> Cheers
> > > > > >>
> > > > > >> On Tue, Feb 3, 2015 at 6:47 AM, sleimanjneidi <
> > > > jneidi.sleiman@gmail.com
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi all,
> > > > > >> > I am using hbase-0.98.1-cdh5.1.4 client and I am a bit
> confused
> > by
> > > > the
> > > > > >> > documentation of HConnection. The document says the following:
> > > > > >> >
> > > > > >> > HConnection instances can be shared. Sharing is usually what
> you
> > > > want
> > > > > >> > because rather than each HConnection instance having to do its
> > own
> > > > > >> > discovery of regions out on the cluster, instead, all clients
> > get
> > > to
> > > > > >> share
> > > > > >> > the one cache of locations. HConnectionManager does the
> sharing
> > > for
> > > > > you
> > > > > >> if
> > > > > >> > you go by it getting connections. Sharing makes cleanup of
> > > > > HConnections
> > > > > >> > awkward. .
> > > > > >> >
> > > > > >> > So now I have a simple question: Can I share the same
> > HConnection
> > > > > >> instance
> > > > > >> > in my entire application?
> > > > > >> > And write some magic code to know when to close or never close
> > at
> > > > all?
> > > > > >> > Or I have to create an instance and close it every time I do a
> > > CRUD
> > > > > >> > operation ?
> > > > > >> >
> > > > > >> > Many thanks
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
Re: Re: managing HConnection
Posted by Sleiman Jneidi <jn...@gmail.com>.
a single HConnection
On Fri, Feb 13, 2015 at 11:12 AM, Serega Sheypak <se...@gmail.com>
wrote:
> What are you trying to achieve?
>
> 2015-02-13 12:36 GMT+03:00 Sleiman Jneidi <jn...@gmail.com>:
>
> > To be honest guys I am still confused, especially that that HConnection
> > implements Closeable and hence everyone has the right to close the
> > connection. I wrote this code to manage connections but I am not sure
> about
> > its correctness.
> >
> >
> > private static class HConnectionProvider {
> >
> > private static HConnection hConnection;
> >
> > private static final Lock LOCK = new ReentrantLock();
> >
> > static {
> >
> > hConnection = createNewConnection();
> >
> > Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
> >
> > @Override
> >
> > public void run() {
> >
> > if(hConnection!=null && !hConnection.isClosed()){
> >
> > try {
> >
> > hConnection.close();
> >
> > } catch (IOException e) {
> >
> > e.printStackTrace();
> >
> > }
> >
> > }
> >
> > }
> >
> > }));
> >
> > }
> >
> > public static HConnection connection(){
> >
> > if(!hConnection.isClosed()){
> >
> > return hConnection;
> >
> > }
> >
> > boolean acquired = false;
> >
> > try{
> >
> > acquired = LOCK.tryLock(5,TimeUnit.SECONDS);
> >
> > if(hConnection.isClosed()){
> >
> > hConnection = createNewConnection();
> >
> > }
> >
> > return hConnection;
> >
> > } catch (InterruptedException e) {
> >
> > throw new RuntimeException(e);
> >
> > }finally{
> >
> > if(acquired){
> >
> > LOCK.unlock();
> >
> > }
> >
> > }
> >
> > }
> >
> > private static HConnection createNewConnection(){
> >
> > try {
> >
> > HConnection connection = HConnectionManager.createConnection(config);
> >
> > return connection;
> >
> > } catch (IOException e) {
> >
> > throw new RuntimeException(e);
> >
> > }
> >
> > }
> >
> > }
> >
> > On Fri, Feb 13, 2015 at 8:57 AM, Serega Sheypak <
> serega.sheypak@gmail.com>
> > wrote:
> >
> > > Hi, really, I can share one Hconnection for the whole application.
> > > It's done by design. I have several servlets. Each servlet has 1-2
> > > controllers working with hbase internally (put/get/e.t.c)
> > > Right now I don't see any reason to refactor code and share single
> > > HConnection for all controllers in servlets.
> > >
> > >
> > > 2015-02-13 6:56 GMT+03:00 David chen <c7...@163.com>:
> > >
> > > > Hi Serega,
> > > > I am very interesting in the reason why per application need to
> create
> > 5
> > > > instead of only one HConnection instances during servlet
> > initialization?
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > At 2015-02-04 01:01:38, "Serega Sheypak" <se...@gmail.com>
> > > wrote:
> > > > >Hi, guys from group helped me a lot. I did solve pretty the same
> > problem
> > > > >(CRUD web-app)
> > > > >
> > > > >1. Use single instance of HConnection per application.
> > > > >2. Instantiate it once.
> > > > >3. create HTable instance for each CRUD operation and safely close
> it
> > > > >(try-catch-finally). Use the same HConnection to create any HTable
> for
> > > > CRUD
> > > > >operation.
> > > > >4. DO NOT close HConnection after CRUD operation
> > > > >
> > > > >I have logic controllers which get HConnection injection in
> > > > >HttpServlet.init method.
> > > > >So I have 5 HConnection instances per application created during
> > servlet
> > > > >initialization
> > > > >
> > > > >
> > > > >2015-02-03 18:12 GMT+03:00 Ted Yu <yu...@gmail.com>:
> > > > >
> > > > >> Please see '61.1. Cluster Connections' under
> > > > >> http://hbase.apache.org/book.html#architecture.client
> > > > >>
> > > > >> Cheers
> > > > >>
> > > > >> On Tue, Feb 3, 2015 at 6:47 AM, sleimanjneidi <
> > > jneidi.sleiman@gmail.com
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Hi all,
> > > > >> > I am using hbase-0.98.1-cdh5.1.4 client and I am a bit confused
> by
> > > the
> > > > >> > documentation of HConnection. The document says the following:
> > > > >> >
> > > > >> > HConnection instances can be shared. Sharing is usually what you
> > > want
> > > > >> > because rather than each HConnection instance having to do its
> own
> > > > >> > discovery of regions out on the cluster, instead, all clients
> get
> > to
> > > > >> share
> > > > >> > the one cache of locations. HConnectionManager does the sharing
> > for
> > > > you
> > > > >> if
> > > > >> > you go by it getting connections. Sharing makes cleanup of
> > > > HConnections
> > > > >> > awkward. .
> > > > >> >
> > > > >> > So now I have a simple question: Can I share the same
> HConnection
> > > > >> instance
> > > > >> > in my entire application?
> > > > >> > And write some magic code to know when to close or never close
> at
> > > all?
> > > > >> > Or I have to create an instance and close it every time I do a
> > CRUD
> > > > >> > operation ?
> > > > >> >
> > > > >> > Many thanks
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
Re: Re: managing HConnection
Posted by Serega Sheypak <se...@gmail.com>.
What are you trying to achieve?
2015-02-13 12:36 GMT+03:00 Sleiman Jneidi <jn...@gmail.com>:
> To be honest guys I am still confused, especially that that HConnection
> implements Closeable and hence everyone has the right to close the
> connection. I wrote this code to manage connections but I am not sure about
> its correctness.
>
>
> private static class HConnectionProvider {
>
> private static HConnection hConnection;
>
> private static final Lock LOCK = new ReentrantLock();
>
> static {
>
> hConnection = createNewConnection();
>
> Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
>
> @Override
>
> public void run() {
>
> if(hConnection!=null && !hConnection.isClosed()){
>
> try {
>
> hConnection.close();
>
> } catch (IOException e) {
>
> e.printStackTrace();
>
> }
>
> }
>
> }
>
> }));
>
> }
>
> public static HConnection connection(){
>
> if(!hConnection.isClosed()){
>
> return hConnection;
>
> }
>
> boolean acquired = false;
>
> try{
>
> acquired = LOCK.tryLock(5,TimeUnit.SECONDS);
>
> if(hConnection.isClosed()){
>
> hConnection = createNewConnection();
>
> }
>
> return hConnection;
>
> } catch (InterruptedException e) {
>
> throw new RuntimeException(e);
>
> }finally{
>
> if(acquired){
>
> LOCK.unlock();
>
> }
>
> }
>
> }
>
> private static HConnection createNewConnection(){
>
> try {
>
> HConnection connection = HConnectionManager.createConnection(config);
>
> return connection;
>
> } catch (IOException e) {
>
> throw new RuntimeException(e);
>
> }
>
> }
>
> }
>
> On Fri, Feb 13, 2015 at 8:57 AM, Serega Sheypak <se...@gmail.com>
> wrote:
>
> > Hi, really, I can share one Hconnection for the whole application.
> > It's done by design. I have several servlets. Each servlet has 1-2
> > controllers working with hbase internally (put/get/e.t.c)
> > Right now I don't see any reason to refactor code and share single
> > HConnection for all controllers in servlets.
> >
> >
> > 2015-02-13 6:56 GMT+03:00 David chen <c7...@163.com>:
> >
> > > Hi Serega,
> > > I am very interesting in the reason why per application need to create
> 5
> > > instead of only one HConnection instances during servlet
> initialization?
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > At 2015-02-04 01:01:38, "Serega Sheypak" <se...@gmail.com>
> > wrote:
> > > >Hi, guys from group helped me a lot. I did solve pretty the same
> problem
> > > >(CRUD web-app)
> > > >
> > > >1. Use single instance of HConnection per application.
> > > >2. Instantiate it once.
> > > >3. create HTable instance for each CRUD operation and safely close it
> > > >(try-catch-finally). Use the same HConnection to create any HTable for
> > > CRUD
> > > >operation.
> > > >4. DO NOT close HConnection after CRUD operation
> > > >
> > > >I have logic controllers which get HConnection injection in
> > > >HttpServlet.init method.
> > > >So I have 5 HConnection instances per application created during
> servlet
> > > >initialization
> > > >
> > > >
> > > >2015-02-03 18:12 GMT+03:00 Ted Yu <yu...@gmail.com>:
> > > >
> > > >> Please see '61.1. Cluster Connections' under
> > > >> http://hbase.apache.org/book.html#architecture.client
> > > >>
> > > >> Cheers
> > > >>
> > > >> On Tue, Feb 3, 2015 at 6:47 AM, sleimanjneidi <
> > jneidi.sleiman@gmail.com
> > > >
> > > >> wrote:
> > > >>
> > > >> > Hi all,
> > > >> > I am using hbase-0.98.1-cdh5.1.4 client and I am a bit confused by
> > the
> > > >> > documentation of HConnection. The document says the following:
> > > >> >
> > > >> > HConnection instances can be shared. Sharing is usually what you
> > want
> > > >> > because rather than each HConnection instance having to do its own
> > > >> > discovery of regions out on the cluster, instead, all clients get
> to
> > > >> share
> > > >> > the one cache of locations. HConnectionManager does the sharing
> for
> > > you
> > > >> if
> > > >> > you go by it getting connections. Sharing makes cleanup of
> > > HConnections
> > > >> > awkward. .
> > > >> >
> > > >> > So now I have a simple question: Can I share the same HConnection
> > > >> instance
> > > >> > in my entire application?
> > > >> > And write some magic code to know when to close or never close at
> > all?
> > > >> > Or I have to create an instance and close it every time I do a
> CRUD
> > > >> > operation ?
> > > >> >
> > > >> > Many thanks
> > > >> >
> > > >> >
> > > >> >
> > > >>
> > >
> >
>
Re: Re: managing HConnection
Posted by Sleiman Jneidi <jn...@gmail.com>.
To be honest guys I am still confused, especially that that HConnection
implements Closeable and hence everyone has the right to close the
connection. I wrote this code to manage connections but I am not sure about
its correctness.
private static class HConnectionProvider {
private static HConnection hConnection;
private static final Lock LOCK = new ReentrantLock();
static {
hConnection = createNewConnection();
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
@Override
public void run() {
if(hConnection!=null && !hConnection.isClosed()){
try {
hConnection.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}));
}
public static HConnection connection(){
if(!hConnection.isClosed()){
return hConnection;
}
boolean acquired = false;
try{
acquired = LOCK.tryLock(5,TimeUnit.SECONDS);
if(hConnection.isClosed()){
hConnection = createNewConnection();
}
return hConnection;
} catch (InterruptedException e) {
throw new RuntimeException(e);
}finally{
if(acquired){
LOCK.unlock();
}
}
}
private static HConnection createNewConnection(){
try {
HConnection connection = HConnectionManager.createConnection(config);
return connection;
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
On Fri, Feb 13, 2015 at 8:57 AM, Serega Sheypak <se...@gmail.com>
wrote:
> Hi, really, I can share one Hconnection for the whole application.
> It's done by design. I have several servlets. Each servlet has 1-2
> controllers working with hbase internally (put/get/e.t.c)
> Right now I don't see any reason to refactor code and share single
> HConnection for all controllers in servlets.
>
>
> 2015-02-13 6:56 GMT+03:00 David chen <c7...@163.com>:
>
> > Hi Serega,
> > I am very interesting in the reason why per application need to create 5
> > instead of only one HConnection instances during servlet initialization?
> >
> >
> >
> >
> >
> >
> >
> >
> > At 2015-02-04 01:01:38, "Serega Sheypak" <se...@gmail.com>
> wrote:
> > >Hi, guys from group helped me a lot. I did solve pretty the same problem
> > >(CRUD web-app)
> > >
> > >1. Use single instance of HConnection per application.
> > >2. Instantiate it once.
> > >3. create HTable instance for each CRUD operation and safely close it
> > >(try-catch-finally). Use the same HConnection to create any HTable for
> > CRUD
> > >operation.
> > >4. DO NOT close HConnection after CRUD operation
> > >
> > >I have logic controllers which get HConnection injection in
> > >HttpServlet.init method.
> > >So I have 5 HConnection instances per application created during servlet
> > >initialization
> > >
> > >
> > >2015-02-03 18:12 GMT+03:00 Ted Yu <yu...@gmail.com>:
> > >
> > >> Please see '61.1. Cluster Connections' under
> > >> http://hbase.apache.org/book.html#architecture.client
> > >>
> > >> Cheers
> > >>
> > >> On Tue, Feb 3, 2015 at 6:47 AM, sleimanjneidi <
> jneidi.sleiman@gmail.com
> > >
> > >> wrote:
> > >>
> > >> > Hi all,
> > >> > I am using hbase-0.98.1-cdh5.1.4 client and I am a bit confused by
> the
> > >> > documentation of HConnection. The document says the following:
> > >> >
> > >> > HConnection instances can be shared. Sharing is usually what you
> want
> > >> > because rather than each HConnection instance having to do its own
> > >> > discovery of regions out on the cluster, instead, all clients get to
> > >> share
> > >> > the one cache of locations. HConnectionManager does the sharing for
> > you
> > >> if
> > >> > you go by it getting connections. Sharing makes cleanup of
> > HConnections
> > >> > awkward. .
> > >> >
> > >> > So now I have a simple question: Can I share the same HConnection
> > >> instance
> > >> > in my entire application?
> > >> > And write some magic code to know when to close or never close at
> all?
> > >> > Or I have to create an instance and close it every time I do a CRUD
> > >> > operation ?
> > >> >
> > >> > Many thanks
> > >> >
> > >> >
> > >> >
> > >>
> >
>
Re: Re: managing HConnection
Posted by Serega Sheypak <se...@gmail.com>.
Hi, really, I can share one Hconnection for the whole application.
It's done by design. I have several servlets. Each servlet has 1-2
controllers working with hbase internally (put/get/e.t.c)
Right now I don't see any reason to refactor code and share single
HConnection for all controllers in servlets.
2015-02-13 6:56 GMT+03:00 David chen <c7...@163.com>:
> Hi Serega,
> I am very interesting in the reason why per application need to create 5
> instead of only one HConnection instances during servlet initialization?
>
>
>
>
>
>
>
>
> At 2015-02-04 01:01:38, "Serega Sheypak" <se...@gmail.com> wrote:
> >Hi, guys from group helped me a lot. I did solve pretty the same problem
> >(CRUD web-app)
> >
> >1. Use single instance of HConnection per application.
> >2. Instantiate it once.
> >3. create HTable instance for each CRUD operation and safely close it
> >(try-catch-finally). Use the same HConnection to create any HTable for
> CRUD
> >operation.
> >4. DO NOT close HConnection after CRUD operation
> >
> >I have logic controllers which get HConnection injection in
> >HttpServlet.init method.
> >So I have 5 HConnection instances per application created during servlet
> >initialization
> >
> >
> >2015-02-03 18:12 GMT+03:00 Ted Yu <yu...@gmail.com>:
> >
> >> Please see '61.1. Cluster Connections' under
> >> http://hbase.apache.org/book.html#architecture.client
> >>
> >> Cheers
> >>
> >> On Tue, Feb 3, 2015 at 6:47 AM, sleimanjneidi <jneidi.sleiman@gmail.com
> >
> >> wrote:
> >>
> >> > Hi all,
> >> > I am using hbase-0.98.1-cdh5.1.4 client and I am a bit confused by the
> >> > documentation of HConnection. The document says the following:
> >> >
> >> > HConnection instances can be shared. Sharing is usually what you want
> >> > because rather than each HConnection instance having to do its own
> >> > discovery of regions out on the cluster, instead, all clients get to
> >> share
> >> > the one cache of locations. HConnectionManager does the sharing for
> you
> >> if
> >> > you go by it getting connections. Sharing makes cleanup of
> HConnections
> >> > awkward. .
> >> >
> >> > So now I have a simple question: Can I share the same HConnection
> >> instance
> >> > in my entire application?
> >> > And write some magic code to know when to close or never close at all?
> >> > Or I have to create an instance and close it every time I do a CRUD
> >> > operation ?
> >> >
> >> > Many thanks
> >> >
> >> >
> >> >
> >>
>
Re:Re: managing HConnection
Posted by David chen <c7...@163.com>.
Hi Serega,
I am very interesting in the reason why per application need to create 5 instead of only one HConnection instances during servlet initialization?
At 2015-02-04 01:01:38, "Serega Sheypak" <se...@gmail.com> wrote:
>Hi, guys from group helped me a lot. I did solve pretty the same problem
>(CRUD web-app)
>
>1. Use single instance of HConnection per application.
>2. Instantiate it once.
>3. create HTable instance for each CRUD operation and safely close it
>(try-catch-finally). Use the same HConnection to create any HTable for CRUD
>operation.
>4. DO NOT close HConnection after CRUD operation
>
>I have logic controllers which get HConnection injection in
>HttpServlet.init method.
>So I have 5 HConnection instances per application created during servlet
>initialization
>
>
>2015-02-03 18:12 GMT+03:00 Ted Yu <yu...@gmail.com>:
>
>> Please see '61.1. Cluster Connections' under
>> http://hbase.apache.org/book.html#architecture.client
>>
>> Cheers
>>
>> On Tue, Feb 3, 2015 at 6:47 AM, sleimanjneidi <jn...@gmail.com>
>> wrote:
>>
>> > Hi all,
>> > I am using hbase-0.98.1-cdh5.1.4 client and I am a bit confused by the
>> > documentation of HConnection. The document says the following:
>> >
>> > HConnection instances can be shared. Sharing is usually what you want
>> > because rather than each HConnection instance having to do its own
>> > discovery of regions out on the cluster, instead, all clients get to
>> share
>> > the one cache of locations. HConnectionManager does the sharing for you
>> if
>> > you go by it getting connections. Sharing makes cleanup of HConnections
>> > awkward. .
>> >
>> > So now I have a simple question: Can I share the same HConnection
>> instance
>> > in my entire application?
>> > And write some magic code to know when to close or never close at all?
>> > Or I have to create an instance and close it every time I do a CRUD
>> > operation ?
>> >
>> > Many thanks
>> >
>> >
>> >
>>
Re: managing HConnection
Posted by Serega Sheypak <se...@gmail.com>.
Hi, guys from group helped me a lot. I did solve pretty the same problem
(CRUD web-app)
1. Use single instance of HConnection per application.
2. Instantiate it once.
3. create HTable instance for each CRUD operation and safely close it
(try-catch-finally). Use the same HConnection to create any HTable for CRUD
operation.
4. DO NOT close HConnection after CRUD operation
I have logic controllers which get HConnection injection in
HttpServlet.init method.
So I have 5 HConnection instances per application created during servlet
initialization
2015-02-03 18:12 GMT+03:00 Ted Yu <yu...@gmail.com>:
> Please see '61.1. Cluster Connections' under
> http://hbase.apache.org/book.html#architecture.client
>
> Cheers
>
> On Tue, Feb 3, 2015 at 6:47 AM, sleimanjneidi <jn...@gmail.com>
> wrote:
>
> > Hi all,
> > I am using hbase-0.98.1-cdh5.1.4 client and I am a bit confused by the
> > documentation of HConnection. The document says the following:
> >
> > HConnection instances can be shared. Sharing is usually what you want
> > because rather than each HConnection instance having to do its own
> > discovery of regions out on the cluster, instead, all clients get to
> share
> > the one cache of locations. HConnectionManager does the sharing for you
> if
> > you go by it getting connections. Sharing makes cleanup of HConnections
> > awkward. .
> >
> > So now I have a simple question: Can I share the same HConnection
> instance
> > in my entire application?
> > And write some magic code to know when to close or never close at all?
> > Or I have to create an instance and close it every time I do a CRUD
> > operation ?
> >
> > Many thanks
> >
> >
> >
>
Re: managing HConnection
Posted by Ted Yu <yu...@gmail.com>.
Please see '61.1. Cluster Connections' under
http://hbase.apache.org/book.html#architecture.client
Cheers
On Tue, Feb 3, 2015 at 6:47 AM, sleimanjneidi <jn...@gmail.com>
wrote:
> Hi all,
> I am using hbase-0.98.1-cdh5.1.4 client and I am a bit confused by the
> documentation of HConnection. The document says the following:
>
> HConnection instances can be shared. Sharing is usually what you want
> because rather than each HConnection instance having to do its own
> discovery of regions out on the cluster, instead, all clients get to share
> the one cache of locations. HConnectionManager does the sharing for you if
> you go by it getting connections. Sharing makes cleanup of HConnections
> awkward. .
>
> So now I have a simple question: Can I share the same HConnection instance
> in my entire application?
> And write some magic code to know when to close or never close at all?
> Or I have to create an instance and close it every time I do a CRUD
> operation ?
>
> Many thanks
>
>
>