You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Shengjun Xin <sx...@gopivotal.com> on 2014/05/05 03:47:02 UTC
Re: Number of hive-server2 threads increments after jdbc connection
Is it same with https://issues.apache.org/jira/browse/HIVE-6866 ?
On Tue, Apr 29, 2014 at 8:26 PM, Dima Fadeyev <df...@pragsis.com> wrote:
> Hi, Chinna. Thanks for your reply.
>
> Yes, modifying code solves the problem. This is what my code looks like (a
> piece of it):
>
> Connection con =
> DriverManager.getConnection("jdbc:hive2://localhost:10000/default", "hive",
> "hive");
> Statement stmt = con.createStatement();
> String tableName = "testHiveDriverTable";
> stmt.execute("drop table if exists " + tableName);
> //stmt.close();
>
> When I uncomment the last line, the number of hive-server2 threads doesn't
> keep incrementing to infinity. However I'm investigating the issue where
> the code is not really my code. Is there a way to correct this behavior
> from within hive-server2 without changing the client's code?
>
> El 29/04/14 14:05, Chinna Rao Lalam escribió:
>
> Hi,
>
> In your code if more connections and statements are created?. If so,
> closed those connections?
> After use close unused connections and statements.
>
>
> Hope It Helps,
> Chinna
>
>
> On Tue, Apr 29, 2014 at 3:47 PM, Dima Fadeyev <df...@pragsis.com>wrote:
>
>> Hello everyone,
>>
>> When I run a jdbc example from
>> https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBCClientSampleCodeagainst my hive server, the number of hive-server2 threads increments. If I
>> execute it long enough I either start seeing exceptions
>>
>> Exception in thread "main" java.sql.SQLException:
>> org.apache.thrift.TApplicationException: Internal error processing
>> ExecuteStatement
>> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:203)
>> at HiveJdbcClient.main(HiveJdbcClient.java:24)
>> Caused by: org.apache.thrift.TApplicationException: Internal error
>> processing ExecuteStatement
>> at
>> org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
>> at
>> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
>> at
>> org.apache.hive.service.cli.thrift.TCLIService$Client.recv_ExecuteStatement(TCLIService.java:213)
>> at
>> org.apache.hive.service.cli.thrift.TCLIService$Client.ExecuteStatement(TCLIService.java:200)
>> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:197)
>> ... 1 more
>>
>> or I bump into a limit of connections with Zookeeper (each hive-server2
>> thread maintains a connection with Zookeeper. I have hive.support.concurrency
>> enabled).
>>
>> In either case I can't connect to hive server after that.
>>
>> I've tried this on hive 0.10 (cdh 4.4) and hive 0.12 (cdh 5.0 and hdp
>> 2.0.6) with same results.
>>
>> Please, could anyone help me resolve this.
>> Thanks in advance.
>>
>>
>>
>
>
> --
> Hope It Helps,
> Chinna
>
>
>
--
Regards
Shengjun
Re: Number of hive-server2 threads increments after jdbc connection
Posted by Dima Fadeyev <df...@pragsis.com>.
Hi,
I have ipc.client.connection.maxidletime set to default (10 sec). The
hive-server2 threads (and tcp connections to zookeeper) stay active
forever. I guess these issues might be related but they are not the same.
Regards,
Dima Fadeyev
El 05/05/14 03:47, Shengjun Xin escribió:
> Is it same with https://issues.apache.org/jira/browse/HIVE-6866 ?
>
>
> On Tue, Apr 29, 2014 at 8:26 PM, Dima Fadeyev <dfadeyev@pragsis.com
> <ma...@pragsis.com>> wrote:
>
> Hi, Chinna. Thanks for your reply.
>
> Yes, modifying code solves the problem. This is what my code looks
> like (a piece of it):
>
> Connection con =
> DriverManager.getConnection("jdbc:hive2://localhost:10000/default", "hive",
> "hive");
> Statement stmt = con.createStatement();
> String tableName = "testHiveDriverTable";
> stmt.execute("drop table if exists " + tableName);
> //stmt.close();
>
> When I uncomment the last line, the number of hive-server2 threads
> doesn't keep incrementing to infinity. However I'm investigating
> the issue where the code is not really my code. Is there a way to
> correct this behavior from within hive-server2 without changing
> the client's code?
>
> El 29/04/14 14:05, Chinna Rao Lalam escribió:
>> Hi,
>>
>> In your code if more connections and statements are created?. If
>> so, closed those connections?
>> After use close unused connections and statements.
>>
>>
>> Hope It Helps,
>> Chinna
>>
>>
>> On Tue, Apr 29, 2014 at 3:47 PM, Dima Fadeyev
>> <dfadeyev@pragsis.com <ma...@pragsis.com>> wrote:
>>
>> Hello everyone,
>>
>> When I run a jdbc example from
>> https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBCClientSampleCode
>> against my hive server, the number of hive-server2 threads
>> increments. If I execute it long enough I either start seeing
>> exceptions
>>
>> Exception in thread "main" java.sql.SQLException:
>> org.apache.thrift.TApplicationException: Internal error
>> processing ExecuteStatement
>> at
>> org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:203)
>> at HiveJdbcClient.main(HiveJdbcClient.java:24)
>> Caused by: org.apache.thrift.TApplicationException: Internal
>> error processing ExecuteStatement
>> at
>> org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
>> at
>> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
>> at
>> org.apache.hive.service.cli.thrift.TCLIService$Client.recv_ExecuteStatement(TCLIService.java:213)
>> at
>> org.apache.hive.service.cli.thrift.TCLIService$Client.ExecuteStatement(TCLIService.java:200)
>> at
>> org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:197)
>> ... 1 more
>>
>> or I bump into a limit of connections with Zookeeper (each
>> hive-server2 thread maintains a connection with Zookeeper. I
>> have hive.support.concurrency enabled).
>>
>> In either case I can't connect to hive server after that.
>>
>> I've tried this on hive 0.10 (cdh 4.4) and hive 0.12 (cdh 5.0
>> and hdp 2.0.6) with same results.
>>
>> Please, could anyone help me resolve this.
>> Thanks in advance.
>>
>>
>>
>>
>>
>> --
>> Hope It Helps,
>> Chinna
>
>
>
>
> --
> Regards
> Shengjun