You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hive.apache.org by Mich Talebzadeh <mi...@gmail.com> on 2016/05/25 20:03:41 UTC

Hive and using Pooled Connections

Hi,


I am sure someone knows the answer to this question.

Does Hive 2.0 use connection pool to connect to its metastore? I see a lot
of open and closed connections to the metastore that may not be necessary.

A *connection pool* is a cache of database connection objects. Connection
pools promote the reuse of connection objects and reduce the number of
times that connection objects are created. Connection pools significantly
improve performance for database-intensive applications because creating
connection objects is costly both in terms of time and resources.
Thanks

Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com

Re: Hive and using Pooled Connections

Posted by Mich Talebzadeh <mi...@gmail.com>.

Thanks Alan.
My Hive is version 2 transactional and its metastore is on Oracle.

I saw this note
<https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions>
stating

.....Using Oracle as the Metastore DB and
"datanucleus.connectionPoolingType=BONECP" may generate intermittent "No
such lock.." and "No such transaction..." errors.  Setting
"datanucleus.connectionPoolingType=DBCP" is recommended in this case....

So I changed the setting to DBCP. Don't know how useful it is going to be.

Regards,



Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com



On 26 May 2016 at 00:05, Alan Gates <al...@gmail.com> wrote:

> It’s depends on how it’s configured.  In $HIVE_HOME/conf/hive-site.xml you
> can set the datanucleus.connectionPoolingType variable to BONECP or DBCP.
> By default it should be using BONECP I believe.  (I think NONE is also a
> valid value, but that doesn’t yet work with ACID turned on.)
>
> Alan.
>
> > On May 25, 2016, at 13:03, Mich Talebzadeh <mi...@gmail.com>
> wrote:
> >
> >
> > Hi,
> >
> >
> > I am sure someone knows the answer to this question.
> >
> > Does Hive 2.0 use connection pool to connect to its metastore? I see a
> lot of open and closed connections to the metastore that may not be
> necessary.
> >
> > A connection pool is a cache of database connection objects. Connection
> pools promote the reuse of connection objects and reduce the number of
> times that connection objects are created. Connection pools significantly
> improve performance for database-intensive applications because creating
> connection objects is costly both in terms of time and resources.
> >
> > Thanks
> >
> > Dr Mich Talebzadeh
> >
> > LinkedIn
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> >
> > http://talebzadehmich.wordpress.com
> >
>
>

Re: Hive and using Pooled Connections

Posted by Alan Gates <al...@gmail.com>.

It’s depends on how it’s configured.  In $HIVE_HOME/conf/hive-site.xml you can set the datanucleus.connectionPoolingType variable to BONECP or DBCP.  By default it should be using BONECP I believe.  (I think NONE is also a valid value, but that doesn’t yet work with ACID turned on.)

Alan.

> On May 25, 2016, at 13:03, Mich Talebzadeh <mi...@gmail.com> wrote:
> 
> 
> Hi,
> 
> 
> I am sure someone knows the answer to this question.
> 
> Does Hive 2.0 use connection pool to connect to its metastore? I see a lot of open and closed connections to the metastore that may not be necessary.
> 
> A connection pool is a cache of database connection objects. Connection pools promote the reuse of connection objects and reduce the number of times that connection objects are created. Connection pools significantly improve performance for database-intensive applications because creating connection objects is costly both in terms of time and resources.
> 
> Thanks
> 
> Dr Mich Talebzadeh
>  
> LinkedIn  https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>  
> http://talebzadehmich.wordpress.com
>