You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Scott Whitecross <sw...@gmail.com> on 2010/08/30 21:50:49 UTC
Having a Connections Leak with the Hive Server
Hi all -
I'm running the Hive Server to allow multiple JDBC queries and inserts over
the course of a day. What I've noticed is that there seems to be a pretty
bad leak with connections over the course of a week, to the point of making
the machine unreachable. Looking through the code being used, it appears to
be closing Hive connections appropriately. Is there a known problem with
Hive and connections now? (I'm currently running Hive .4).
Thanks.
RE: Simple question FROM clause
Posted by Ashish Thusoo <at...@facebook.com>.
You have to create a dummy table. I think there is a JIRA open to provide a usage similar to the one you mentioned below.
Ashish
-----Original Message-----
From: elein [mailto:elein@varlena.com]
Sent: Monday, August 30, 2010 3:16 PM
To: hive-user@hadoop.apache.org
Cc: elein
Subject: Simple question FROM clause
I would like to select some constants and there is no from clause, but hive says it needs one. Do I have to create a dummy table for these or is there a better way?
select 'this', 'and', 'that';
elein
elein@varlena.com
Simple question FROM clause
Posted by elein <el...@varlena.com>.
I would like to select some constants and there is no from clause,
but hive says it needs one. Do I have to create a dummy table for
these or is there a better way?
select 'this', 'and', 'that';
elein
elein@varlena.com
RE: Having a Connections Leak with the Hive Server
Posted by Bennie Schut <bs...@ebuddy.com>.
We had some leaking file descriptors which ended up being a problem in hadoop. They fixed it on 0.21 but not on older versions. There is a workaround for hive which we successfully use. By adding this to your hive-site.xml:
<!-- workaround for connection leak problem fixed in HADOOP-5476 but only commited to hadoop 0.21.0 -->
<property>
<name>hive.fileformat.check</name>
<value>false</value>
</property>
Bennie.
________________________________
From: Dave Brondsema [mailto:dbrondsema@geek.net]
Sent: Thursday, September 02, 2010 3:13 PM
To: hive-user@hadoop.apache.org
Subject: Re: Having a Connections Leak with the Hive Server
Scott, after re-reading your original email, I'm thinking maybe we didn't have the same problem. Hive crashed for us when it ran out of file descriptors, it didn't hang. Nonetheless, an upgrade may help.
On Wed, Sep 1, 2010 at 10:24 AM, Scott Whitecross <sw...@gmail.com>> wrote:
Thanks Dave. We've been investigating moving to .6, and this may help make the decision easier.
On Tue, Aug 31, 2010 at 5:02 PM, Dave Brondsema <db...@geek.net>> wrote:
We had that problem on 0.4 also. 0.6 seems to be working better now, but we only switched a day or two ago. See https://issues.apache.org/jira/browse/HIVE-1181 and http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/201001.mbox/%3CD35D2F55-D770-422D-B418-6BEE212A610C@forward.co.uk%3E
On Mon, Aug 30, 2010 at 3:50 PM, Scott Whitecross <sw...@gmail.com>> wrote:
Hi all -
I'm running the Hive Server to allow multiple JDBC queries and inserts over the course of a day. What I've noticed is that there seems to be a pretty bad leak with connections over the course of a week, to the point of making the machine unreachable. Looking through the code being used, it appears to be closing Hive connections appropriately. Is there a known problem with Hive and connections now? (I'm currently running Hive .4).
Thanks.
--
Dave Brondsema
Software Engineer
Geeknet
www.geek.net<http://www.geek.net>
--
Dave Brondsema
Software Engineer
Geeknet
www.geek.net<http://www.geek.net>
Re: Having a Connections Leak with the Hive Server
Posted by Dave Brondsema <db...@geek.net>.
Scott, after re-reading your original email, I'm thinking maybe we didn't
have the same problem. Hive crashed for us when it ran out of file
descriptors, it didn't hang. Nonetheless, an upgrade may help.
On Wed, Sep 1, 2010 at 10:24 AM, Scott Whitecross <sw...@gmail.com>wrote:
> Thanks Dave. We've been investigating moving to .6, and this may help
> make the decision easier.
>
>
> On Tue, Aug 31, 2010 at 5:02 PM, Dave Brondsema <db...@geek.net>wrote:
>
>> We had that problem on 0.4 also. 0.6 seems to be working better now, but
>> we only switched a day or two ago. See
>> https://issues.apache.org/jira/browse/HIVE-1181 and
>> http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/201001.mbox/%3CD35D2F55-D770-422D-B418-6BEE212A610C@forward.co.uk%3E
>>
>>
>> On Mon, Aug 30, 2010 at 3:50 PM, Scott Whitecross <sw...@gmail.com>wrote:
>>
>>> Hi all -
>>>
>>> I'm running the Hive Server to allow multiple JDBC queries and inserts
>>> over the course of a day. What I've noticed is that there seems to be a
>>> pretty bad leak with connections over the course of a week, to the point of
>>> making the machine unreachable. Looking through the code being used, it
>>> appears to be closing Hive connections appropriately. Is there a known
>>> problem with Hive and connections now? (I'm currently running Hive .4).
>>>
>>> Thanks.
>>>
>>
>>
>>
>> --
>> Dave Brondsema
>> Software Engineer
>> Geeknet
>>
>> www.geek.net
>>
>
>
--
Dave Brondsema
Software Engineer
Geeknet
www.geek.net
Re: Having a Connections Leak with the Hive Server
Posted by Scott Whitecross <sw...@gmail.com>.
Thanks Dave. We've been investigating moving to .6, and this may help make
the decision easier.
On Tue, Aug 31, 2010 at 5:02 PM, Dave Brondsema <db...@geek.net> wrote:
> We had that problem on 0.4 also. 0.6 seems to be working better now, but
> we only switched a day or two ago. See
> https://issues.apache.org/jira/browse/HIVE-1181 and
> http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/201001.mbox/%3CD35D2F55-D770-422D-B418-6BEE212A610C@forward.co.uk%3E
>
>
> On Mon, Aug 30, 2010 at 3:50 PM, Scott Whitecross <sw...@gmail.com>wrote:
>
>> Hi all -
>>
>> I'm running the Hive Server to allow multiple JDBC queries and inserts
>> over the course of a day. What I've noticed is that there seems to be a
>> pretty bad leak with connections over the course of a week, to the point of
>> making the machine unreachable. Looking through the code being used, it
>> appears to be closing Hive connections appropriately. Is there a known
>> problem with Hive and connections now? (I'm currently running Hive .4).
>>
>> Thanks.
>>
>
>
>
> --
> Dave Brondsema
> Software Engineer
> Geeknet
>
> www.geek.net
>
Re: Having a Connections Leak with the Hive Server
Posted by Dave Brondsema <db...@geek.net>.
We had that problem on 0.4 also. 0.6 seems to be working better now, but we
only switched a day or two ago. See
https://issues.apache.org/jira/browse/HIVE-1181 and
http://mail-archives.apache.org/mod_mbox/hadoop-hive-user/201001.mbox/%3CD35D2F55-D770-422D-B418-6BEE212A610C@forward.co.uk%3E
On Mon, Aug 30, 2010 at 3:50 PM, Scott Whitecross <sw...@gmail.com>wrote:
> Hi all -
>
> I'm running the Hive Server to allow multiple JDBC queries and inserts over
> the course of a day. What I've noticed is that there seems to be a pretty
> bad leak with connections over the course of a week, to the point of making
> the machine unreachable. Looking through the code being used, it appears to
> be closing Hive connections appropriately. Is there a known problem with
> Hive and connections now? (I'm currently running Hive .4).
>
> Thanks.
>
--
Dave Brondsema
Software Engineer
Geeknet
www.geek.net