You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Kashif Hussain <ka...@gmail.com> on 2016/01/12 08:16:50 UTC
Maximum number of concurrent hive connections
Hi,
I need 3000-4000 concurrent connections to hive.The hive metastore is
running on a 256 GB Ram machine.
Can anyone tell me what's the maximum number of connections a hive
metstaore can support ?
Regards,
Kashif
RE: Maximum number of concurrent hive connections
Posted by "Singh, Abhijeet" <ab...@informatica.com>.
MySQL can support 4K connections but that might be it. The machine should be used only for this purpose.
From: Kashif Hussain [mailto:kash.time@gmail.com]
Sent: Tuesday, January 12, 2016 12:54 PM
To: user@hive.apache.org
Subject: Re: Maximum number of concurrent hive connections
It's on MySQL.
I am making connections through hiveserver2.
On Tue, Jan 12, 2016 at 12:50 PM, Mich Talebzadeh <mi...@peridale.co.uk>> wrote:
Depends on metastore settings. What is your metastore on? Oracle, SAP etc
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com<http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Kashif Hussain [mailto:kash.time@gmail.com<ma...@gmail.com>]
Sent: 12 January 2016 07:17
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Maximum number of concurrent hive connections
Hi,
I need 3000-4000 concurrent connections to hive.The hive metastore is running on a 256 GB Ram machine.
Can anyone tell me what's the maximum number of connections a hive metstaore can support ?
Regards,
Kashif
RE: Maximum number of concurrent hive connections
Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
The connections are made through hive server JDBC thin client and Hive server will not open new connections to Metastore for new clients. The number of connections to metastore remains pretty the same. I think you will have constraint on the number of containers that can run concurrently.
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Kashif Hussain [mailto:kash.time@gmail.com]
Sent: 12 January 2016 07:24
To: user@hive.apache.org
Subject: Re: Maximum number of concurrent hive connections
It's on MySQL.
I am making connections through hiveserver2.
On Tue, Jan 12, 2016 at 12:50 PM, Mich Talebzadeh <mich@peridale.co.uk <ma...@peridale.co.uk> > wrote:
Depends on metastore settings. What is your metastore on? Oracle, SAP etc
Dr Mich Talebzadeh
LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Kashif Hussain [mailto:kash.time@gmail.com <ma...@gmail.com> ]
Sent: 12 January 2016 07:17
To: user@hive.apache.org <ma...@hive.apache.org>
Subject: Maximum number of concurrent hive connections
Hi,
I need 3000-4000 concurrent connections to hive.The hive metastore is running on a 256 GB Ram machine.
Can anyone tell me what's the maximum number of connections a hive metstaore can support ?
Regards,
Kashif
Re: Maximum number of concurrent hive connections
Posted by Kashif Hussain <ka...@gmail.com>.
It's on MySQL.
I am making connections through hiveserver2.
On Tue, Jan 12, 2016 at 12:50 PM, Mich Talebzadeh <mi...@peridale.co.uk>
wrote:
> Depends on metastore settings. What is your metastore on? Oracle, SAP etc
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> *Sybase ASE 15 Gold Medal Award 2008*
>
> A Winning Strategy: Running the most Critical Financial Data on ASE 15
>
>
> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE
> 15", ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Complex Event Processing in Heterogeneous Environments*, ISBN:
> 978-0-9563693-3-8
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Technology
> Ltd, its subsidiaries or their employees, unless expressly so stated. It is
> the responsibility of the recipient to ensure that this email is virus
> free, therefore neither Peridale Ltd, its subsidiaries nor their employees
> accept any responsibility.
>
>
>
> *From:* Kashif Hussain [mailto:kash.time@gmail.com]
> *Sent:* 12 January 2016 07:17
> *To:* user@hive.apache.org
> *Subject:* Maximum number of concurrent hive connections
>
>
>
> Hi,
>
>
>
> I need 3000-4000 concurrent connections to hive.The hive metastore is
> running on a 256 GB Ram machine.
>
> Can anyone tell me what's the maximum number of connections a hive
> metstaore can support ?
>
>
>
>
>
> Regards,
>
> Kashif
>
RE: Maximum number of concurrent hive connections
Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
Depends on metastore settings. What is your metastore on? Oracle, SAP etc
Dr Mich Talebzadeh
LinkedIn <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
Sybase ASE 15 Gold Medal Award 2008
A Winning Strategy: Running the most Critical Financial Data on ASE 15
<http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf
Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.
co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4
Publications due shortly:
Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8
Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly
<http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com
NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
From: Kashif Hussain [mailto:kash.time@gmail.com]
Sent: 12 January 2016 07:17
To: user@hive.apache.org
Subject: Maximum number of concurrent hive connections
Hi,
I need 3000-4000 concurrent connections to hive.The hive metastore is running on a 256 GB Ram machine.
Can anyone tell me what's the maximum number of connections a hive metstaore can support ?
Regards,
Kashif