You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Kevin Risden <kr...@apache.org> on 2018/10/15 19:11:08 UTC
HiveServer2 performance references?
Are there any resources on HiveServer2 performance - specifically binary vs
HTTP mode? I would also be interested in any information about enabling
compression between HiveServer2 and a client like beeline.
I was looking at HiveServer2 performance going through Knox in KNOX-1524
and found that HTTP mode is significantly slower. I searched and didn't
find any references to performance of HiveServer2 out of the box. I also
didn't see any information about reducing amount of data over the wire with
compression.
Any pointers would be very helpful. Thanks!
Kevin Risden
Re: Unsubscribe
Posted by Lefty Leverenz <le...@gmail.com>.
Richard, to unsubscribe please send a message to
user-unsubscribe@hive.apache.org as described here: Mailing Lists
<http://hive.apache.org/mailing_lists.html>.
Thanks. -- Lefty
On Tue, Oct 16, 2018 at 2:20 PM Richard A. Bross <rb...@oaktreepeak.com> wrote:
> Unsubscribe
>
> ----- Original Message -----
> From: "Kevin Risden" <kr...@apache.org>
> To: user@hive.apache.org
> Sent: Tuesday, October 16, 2018 9:44:17 AM
> Subject: Re: HiveServer2 performance references?
>
>
> Thanks for the responses. I'll wait to test with HIVE-20621 to see how
> much that helps. For HIVE-17194, doesn't look like that is documented
> outside of JIRA currently. It looks like I may have picked up over the wire
> compression by default. I had assumed it wasn't compressed.
>
>
>
> Kevin Risden
>
>
>
>
> On Tue, Oct 16, 2018 at 12:44 AM Gopal Vijayaraghavan < gopalv@apache.org
> > wrote:
>
>
>
>
>
>
> Hi,
>
>
> > I was looking at HiveServer2 performance going through Knox in KNOX-1524
> and found that HTTP mode is significantly slower.
>
>
>
> The HTTP mode does re-auth for every row before HIVE-20621 was fixed –
> Knox should be doing cookie-auth to prevent ActiveDirectory/LDAP from
> throttling this.
>
> I fixed it for the open-source JDBC drivers, but I think the proprietary
> ODBC drivers are still broken (the C++ code doesn’t save cookies).
>
> > I also didn't see any information about reducing amount of data over the
> wire with compression.
>
> It’s possible Knox doesn’t generate/forward the Accept-Encoding headers.
>
>
>
> https://issues.apache.org/jira/browse/HIVE-17194
>
>
> Cheers,
>
> Gopal
>
>
>
>
> From: Prasanth Jayachandran < pjayachandran@hortonworks.com >
> Reply-To: " user@hive.apache.org " < user@hive.apache.org >
> Date: Monday, October 15, 2018 at 12:34 PM
> To: " user@hive.apache.org " < user@hive.apache.org >, "
> user@hive.apache.org " < user@hive.apache.org >
> Subject: Re: HiveServer2 performance references?
>
>
>
>
>
>
>
> One performance fix that is fixed recently and is related is HIVE-20621
>
>
>
>
>
>
> Thanks
>
>
> Prasanth
>
>
>
>
>
>
> On Mon, Oct 15, 2018 at 12:11 PM -0700, "
>
>
>
> Kevin Risden " < krisden@apache.org > wrote:
>
>
>
>
>
>
>
>
>
> Are there any resources on HiveServer2 performance - specifically binary
> vs HTTP mode? I would also be interested in any information about enabling
> compression between HiveServer2 and a client like beeline.
>
>
>
>
>
> I was looking at HiveServer2 performance going through Knox in KNOX-1524
> and found that HTTP mode is significantly slower. I searched and didn't
> find any references to performance of HiveServer2 out of the box. I also
> didn't see any information about reducing amount of data over the wire with
> compression.
>
>
>
>
>
> Any pointers would be very helpful. Thanks!
>
>
>
>
>
>
>
> Kevin Risden
>
Unsubscribe
Posted by "Richard A. Bross" <rb...@oaktreepeak.com>.
Unsubscribe
----- Original Message -----
From: "Kevin Risden" <kr...@apache.org>
To: user@hive.apache.org
Sent: Tuesday, October 16, 2018 9:44:17 AM
Subject: Re: HiveServer2 performance references?
Thanks for the responses. I'll wait to test with HIVE-20621 to see how much that helps. For HIVE-17194, doesn't look like that is documented outside of JIRA currently. It looks like I may have picked up over the wire compression by default. I had assumed it wasn't compressed.
Kevin Risden
On Tue, Oct 16, 2018 at 12:44 AM Gopal Vijayaraghavan < gopalv@apache.org > wrote:
Hi,
> I was looking at HiveServer2 performance going through Knox in KNOX-1524 and found that HTTP mode is significantly slower.
The HTTP mode does re-auth for every row before HIVE-20621 was fixed – Knox should be doing cookie-auth to prevent ActiveDirectory/LDAP from throttling this.
I fixed it for the open-source JDBC drivers, but I think the proprietary ODBC drivers are still broken (the C++ code doesn’t save cookies).
> I also didn't see any information about reducing amount of data over the wire with compression.
It’s possible Knox doesn’t generate/forward the Accept-Encoding headers.
https://issues.apache.org/jira/browse/HIVE-17194
Cheers,
Gopal
From: Prasanth Jayachandran < pjayachandran@hortonworks.com >
Reply-To: " user@hive.apache.org " < user@hive.apache.org >
Date: Monday, October 15, 2018 at 12:34 PM
To: " user@hive.apache.org " < user@hive.apache.org >, " user@hive.apache.org " < user@hive.apache.org >
Subject: Re: HiveServer2 performance references?
One performance fix that is fixed recently and is related is HIVE-20621
Thanks
Prasanth
On Mon, Oct 15, 2018 at 12:11 PM -0700, "
Kevin Risden " < krisden@apache.org > wrote:
Are there any resources on HiveServer2 performance - specifically binary vs HTTP mode? I would also be interested in any information about enabling compression between HiveServer2 and a client like beeline.
I was looking at HiveServer2 performance going through Knox in KNOX-1524 and found that HTTP mode is significantly slower. I searched and didn't find any references to performance of HiveServer2 out of the box. I also didn't see any information about reducing amount of data over the wire with compression.
Any pointers would be very helpful. Thanks!
Kevin Risden
Re: HiveServer2 performance references?
Posted by Kevin Risden <kr...@apache.org>.
Thanks for the responses. I'll wait to test with HIVE-20621 to see how much
that helps. For HIVE-17194, doesn't look like that is documented outside of
JIRA currently. It looks like I may have picked up over the wire
compression by default. I had assumed it wasn't compressed.
Kevin Risden
On Tue, Oct 16, 2018 at 12:44 AM Gopal Vijayaraghavan <go...@apache.org>
wrote:
> Hi,
>
>
> > I was looking at HiveServer2 performance going through Knox in KNOX-1524
> and found that HTTP mode is significantly slower.
>
> The HTTP mode does re-auth for every row before HIVE-20621 was fixed –
> Knox should be doing cookie-auth to prevent ActiveDirectory/LDAP from
> throttling this.
>
> I fixed it for the open-source JDBC drivers, but I think the proprietary
> ODBC drivers are still broken (the C++ code doesn’t save cookies).
>
> > I also didn't see any information about reducing amount of data over the
> wire with compression.
>
> It’s possible Knox doesn’t generate/forward the Accept-Encoding headers.
>
>
>
> https://issues.apache.org/jira/browse/HIVE-17194
>
>
> Cheers,
>
> Gopal
>
>
>
> *From: *Prasanth Jayachandran <pj...@hortonworks.com>
> *Reply-To: *"user@hive.apache.org" <us...@hive.apache.org>
> *Date: *Monday, October 15, 2018 at 12:34 PM
> *To: *"user@hive.apache.org" <us...@hive.apache.org>, "user@hive.apache.org"
> <us...@hive.apache.org>
> *Subject: *Re: HiveServer2 performance references?
>
>
>
> One performance fix that is fixed recently and is related is HIVE-20621
>
>
>
> Thanks
>
> Prasanth
>
>
>
> On Mon, Oct 15, 2018 at 12:11 PM -0700, "
> Kevin Risden
> " <kr...@apache.org> wrote:
>
> Are there any resources on HiveServer2 performance - specifically binary
> vs HTTP mode? I would also be interested in any information about enabling
> compression between HiveServer2 and a client like beeline.
>
>
>
> I was looking at HiveServer2 performance going through Knox in KNOX-1524
> and found that HTTP mode is significantly slower. I searched and didn't
> find any references to performance of HiveServer2 out of the box. I also
> didn't see any information about reducing amount of data over the wire with
> compression.
>
>
>
> Any pointers would be very helpful. Thanks!
>
>
>
> Kevin Risden
>
>
Re: HiveServer2 performance references?
Posted by Gopal Vijayaraghavan <go...@apache.org>.
Hi,
> I was looking at HiveServer2 performance going through Knox in KNOX-1524 and found that HTTP mode is significantly slower.
The HTTP mode does re-auth for every row before HIVE-20621 was fixed – Knox should be doing cookie-auth to prevent ActiveDirectory/LDAP from throttling this.
I fixed it for the open-source JDBC drivers, but I think the proprietary ODBC drivers are still broken (the C++ code doesn’t save cookies).
> I also didn't see any information about reducing amount of data over the wire with compression.
It’s possible Knox doesn’t generate/forward the Accept-Encoding headers.
https://issues.apache.org/jira/browse/HIVE-17194
Cheers,
Gopal
From: Prasanth Jayachandran <pj...@hortonworks.com>
Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
Date: Monday, October 15, 2018 at 12:34 PM
To: "user@hive.apache.org" <us...@hive.apache.org>, "user@hive.apache.org" <us...@hive.apache.org>
Subject: Re: HiveServer2 performance references?
One performance fix that is fixed recently and is related is HIVE-20621
Thanks
Prasanth
On Mon, Oct 15, 2018 at 12:11 PM -0700, "Kevin Risden" <kr...@apache.org> wrote:
Are there any resources on HiveServer2 performance - specifically binary vs HTTP mode? I would also be interested in any information about enabling compression between HiveServer2 and a client like beeline.
I was looking at HiveServer2 performance going through Knox in KNOX-1524 and found that HTTP mode is significantly slower. I searched and didn't find any references to performance of HiveServer2 out of the box. I also didn't see any information about reducing amount of data over the wire with compression.
Any pointers would be very helpful. Thanks!
Kevin Risden
Re: HiveServer2 performance references?
Posted by Prasanth Jayachandran <pj...@hortonworks.com>.
One performance fix that is fixed recently and is related is HIVE-20621
Thanks
Prasanth
On Mon, Oct 15, 2018 at 12:11 PM -0700, "Kevin Risden" <kr...@apache.org>> wrote:
Are there any resources on HiveServer2 performance - specifically binary vs HTTP mode? I would also be interested in any information about enabling compression between HiveServer2 and a client like beeline.
I was looking at HiveServer2 performance going through Knox in KNOX-1524 and found that HTTP mode is significantly slower. I searched and didn't find any references to performance of HiveServer2 out of the box. I also didn't see any information about reducing amount of data over the wire with compression.
Any pointers would be very helpful. Thanks!
Kevin Risden