0.13.0 Hive Issue - Hive client connecting via thrift

[Bryan Jeffrey <br...@gmail.com>]

Hello.

We are running Hadoop 2.2.0, Hive 0.12.0.  We're seeing an error reading
integer fields from ORC tables that looks very similar to HIVE-5991.  In an
attempt to fix the issue, I downloaded Hive 0.13.0 trunk and compiled it.
 On starting Hive 0.13.0, I see the following messages:

2014-02-12 14:08:18,077 INFO  service.HiveServer
(HiveServer.java:main(696)) - Starting hive server on port 10001 with 100
min worker threads and 2147483647 max worker threads
2014-02-12 14:08:18,081 INFO  service.HiveServer
(HiveServer.java:main(698)) - TCP keepalive = true
2014-02-12 14:08:55,998 INFO  metastore.HiveMetaStore
(HiveMetaStore.java:newRawStore(440)) - 0: Opening raw store with
implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2014-02-12 14:08:56,044 INFO  metastore.ObjectStore
(ObjectStore.java:initialize(245)) - ObjectStore, initialize called
2014-02-12 14:08:58,531 INFO  metastore.ObjectStore
(ObjectStore.java:getPMF(314)) - Setting MetaStore object pin classes with
hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
2014-02-12 14:09:02,513 INFO  metastore.ObjectStore
(ObjectStore.java:setConf(228)) - Initialized ObjectStore
2014-02-12 14:09:03,337 WARN  metastore.HiveMetaStore
(HiveMetaStore.java:createDefaultRolesNAddUsers(495)) - Configured auth
manager
org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider
doesn't implement hive.security.authenticator.manager admin role will not
be created.
2014-02-12 14:09:03,338 WARN  metastore.HiveMetaStore
(HiveMetaStore.java:createDefaultRolesNAddUsers(495)) - Configured auth
manager
org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider
doesn't implement hive.security.authenticator.manager admin role will not
be created.
2014-02-12 14:09:03,592 INFO  hive.metastore
(HiveMetaStoreClient.java:open(256)) - Trying to connect to metastore with
URI thrift://localhost:10001
2014-02-12 14:09:03,593 INFO  hive.metastore
(HiveMetaStoreClient.java:open(256)) - Trying to connect to metastore with
URI thrift://localhost:10001

These messages continue to repeat until we're met with a 'Too many open
files' error after several thousand connections are made & files are
opened.

The thrift configuration worked with the older (0.12.0) Hive installation.
 Here is the output of my hive-site.xml:

  <configuration>

<!-- Hive Configuration can either be stored in this file or in the hadoop
configuration files  -->
<!-- that are implied by Hadoop setup variables.
                     -->
<!-- Aside from Hadoop setup variables - this file is provided as a
convenience so that Hive    -->
<!-- users do not have to edit hadoop configuration files (that may be
managed as a centralized -->
<!-- resource).
                    -->


<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <!-- note: variable substituion not working here because it's loaded by
jdo, not Hive -->

<value>jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true</value>
</property>

<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
</property>

<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
</property>

<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>iatf_admin</value>
</property>

<property>
  <!--  this should eventually be deprecated since the metastore should
supply this -->
  <name>hive.metastore.warehouse.dir</name>
  <value>/hive/warehouse</value>
  <description></description>
</property>

<property>
  <name>hive.metastore.uris</name>
  <value>thrift://localhost:10001</value>
</property>

<property>
  <name>hive.metastore.execute.setugi</name>
  <value>true</value>
</property>
</configuration>

Any ideas as to why the client is continually connecting & never closing
open sockets?

Regards,

Bryan Jeffrey