Can't get templeton Hive to work

[Peter Marron <Pe...@trilliumsoftware.com>]

Hi,

I've been trying to get Templeton to work and I have a problem.

I'm using Hadoop 1.0.4 and version 0.11.0 of Hive with the HCatalog
release included. I've tried to follow the installation instructions fairly
closely but I've obviously done something wrong. I can get most things
to work. So I am able to successfully do things like the following:


-- List tables
curl hpcluster1:50111/templeton/v1/ddl/database/default/table?user.name=pmarron
-- Delete a table
curl -s -X DELETE http://hpcluster1:50111/templeton/v1/ddl/database/default/table/test_table?user.name=pmarron
-- Create a table
curl -s -X PUT -HContent-type:application/json -d @createTable.txt http://hpcluster1:50111/templeton/v1/ddl/database/default/table/test_table?user.name=pmarron
-- Describe a table
curl hpcluster1:50111/templeton/v1/ddl/database/default/table/test_table?user.name=pmarron
-- Columns
curl hpcluster1:50111/templeton/v1/ddl/database/default/table/test_table/column?user.name=pmarron
-- Table Properties
curl hpcluster1:50111/templeton/v1/ddl/database/default/table/test_table/property?user.name=pmarron

These all work without problem. Similarly I have been able to launch a Map/Reduce job using
curl -s -d user.name=pmarron -d jar=hadoop-examples-1.0.4.jar wordcount.jar -d class=wordcount -d arg=ordinals.txt -d arg=output 'http://localhost:50111/templeton/v1/mapreduce/jar'

I can also use hcat. So, for example, this works
hcat -e "show tables"

However if I try to access Hive through templeton I get an error:
curl -d user.name=pmarron -d execute="show tables" -d statusdir=fred http://hpcluster1:50111/templeton/v1/hive
will eventually write this error to hdfs://fred/stderr

WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Logging initialized using configuration in jar:file:/home/hadoop/local/taskTracker/distcache/-7420519247952810286_-1791958366_1973852064/hpcluster1/apps/templeton/hive-0.10.0.tar.gz/hive-0.10.0/lib/hive-common-0.10.0.jar!/hive-log4j.properties
Hive history file=/tmp/pmarron/hive_job_log_pmarron_201306250933_259550864.txt
FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

I can't find anything in any of the logs that's in anyway helpful.
(The specified history file /tmp/pmarron/hive_job_log_pmarron_201306250933_259550864.txt doesn't exist.)
However it seems pretty clear that the problem is that Templeton can't connect to the thrift
server. Because if I kill my thrift server I get exactly the same error. I guess the real problem
is that I don't know how templeton knows how to connect to the thrift sever. Googling suggests
that I have to make sure that the "hive.metastore.uris" in the hive-site.xml is what's used,
but I have set this up to the correct value and it doesn't make any difference.

        <property>
        <name>hive.metastore.uris</name>
        <value>thrift://hpcluster1:10000</value>
        </property>

If anyone could suggest what I'm doing wrong I'd be most grateful.
I know that it's probably something very simple...

In passing I should mention that I had a problem
installing hcatalog from the Hive 0.11.0 build.
This is because there is a missing
share/doc/hcatalog/NOTICE.txt file
which the installation explicitly looks for.

Also the entries in the
hcatalog/etc/webhcat/webhcat-default.xml
file which refer to Hive are all set to refer to
hive-0.10.0. Isn't this a weird version of hive
to configure given that this is part of release 0.11.0 of Hive?

Peter Marron
Trillium Software UK Limited

Tel : +44 (0) 118 940 7609
Fax : +44 (0) 118 940 7699
E: Peter.Marron@TrilliumSoftware.com<ma...@TrilliumSoftware.com>