HiveODBC build error

[Paul Trout <pa...@returnpath.net>]

25 Feb 2011

We're interested in using the HiveODBC interface, so I've been trying to build it.  I'm using Hadoop 0.20.2+320, from Cloudera, with Hive 0.7.0CDH3B4.  Initially I was trying this with Hive 0.5.0+32, but we intend to upgrade to CDH3B4 very soon, so I decided to try with the newer version.  When I attempt to build the hiveClient:

cd /usr/local/hive-0.7.0CDH3B4/src
ant clean
ant compile-cpp -Dthrift.home=/usr/local/thrift -Dboost.home=/usr/local/boost_1_45_0

The build fails with:

compile-cpp:
     [exec] mkdir -p /usr/local/hive-0.7.0-CDH3B4/src/build/metastore/objs
     [exec] g++ -Wall -g -fPIC -m64 -DARCH64 -I/usr/local/thrift/include/thrift -I/usr/local/thrift/include/thrift/fb303 -I/usr/local/boost_1_45_0/include -I/usr/local/hive-0.7.0-CDH3B4/src/service/src/gen/thrift/gen-cpp -I/usr/local/hive-0.7.0-CDH3B4/src/ql/src/gen/thrift/gen-cpp -I/usr/local/hive-0.7.0-CDH3B4/src/metastore/src/gen/thrift/gen-cpp -I/usr/local/hive-0.7.0-CDH3B4/src/odbc/src/cpp -c /usr/local/hive-0.7.0-CDH3B4/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp -o /usr/local/hive-0.7.0-CDH3B4/src/build/metastore/objs/ThriftHiveMetastore.o
     [exec] /usr/local/hive-0.7.0-CDH3B4/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp: In member function \u2018virtual bool Apache::Hadoop::Hive::ThriftHiveMetastoreProcessor::process_fn(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::string&, int32_t)\u2019:
     [exec] /usr/local/hive-0.7.0-CDH3B4/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:16677: error: no matching function for call to \u2018Apache::Hadoop::Hive::ThriftHiveMetastoreProcessor::process_fn(apache::thrift::protocol::TProtocol*&, apache::thrift::protocol::TProtocol*&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int32_t&)\u2019
     [exec] /usr/local/thrift/include/thrift/fb303/FacebookService.h:1239: note: candidates are: virtual bool facebook::fb303::FacebookServiceProcessor::process_fn(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::string&, int32_t, void*)
     [exec] make: *** [/usr/local/hive-0.7.0-CDH3B4/src/build/metastore/objs/ThriftHiveMetastore.o] Error 1

To get to this stage, these are the steps I followed (I started with the instructions here: http://wiki.apache.org/hadoop/Hive/HiveODBC):

UnixODBC Build/Install (To confirm it builds prior to trying it with the HiveODBC pieces)
* tar -xzvf unixODBC-2.3.0.tar.gz
* cd unixODBC-2.3.0
* ./configure
* make
* sudo make install
* installed in /usr/local 

Boost Build/Install
* cd /usr/local
* sudo tar -xjvf /buildloc/boost_1_45_0.tar.bz2
* cd boost_1_45_0
* sudo ./bootstrap.sh
* sudo ./bjam
* sudo ./bjam install
* export BOOST_HOME=/usr/local/boost_1_45_0

Installed JDK 1.6.0_17 in /usr/java/jdk1.6.0_17
export JAVA_HOME=/usr/java/jdk1.6.0_17
export HADOOP_HOME=/usr/local/hadoop

Apache Ant 1.8.2 Install
* cd /usr/local
* sudo tar -xzvf /buildloc/apache-ant-1.8.2.tar.gz
* export PATH=/usr/local/apache-ant-1.8.2/bin:$PATH
* export ANT_HOME=/usr/local/apache-ant-1.8.2

Apache Ivy 2.2.0 Install
* cd /usr/local
* sudo tar -xzvf /buildloc/apache-ivy-2.2.0-bin.tar.gz
* cd apache-ivy
* sudo cp ivy-2.2.0.jar /usr/local/apache-ant-1.8.2/lib
* export IVY_HOME=/usr/local/apache-ivy-2.2.0/

Install GNU autoconf 2.68
* cd /buildloc
* tar -xzvf autoconf-2.68.tar.gz
* cd autoconf-2.68
* ./configure --prefix=/usr/local/autoconf-2.68
* make
* make install
* export PATH=/usr/local/autoconf-2.68/bin:$PATH

Thrift Build/Install
* cd /buildloc
* rm -rf thrift-0.6.0
* tar -xzvf thrift-0.6.0.tar.gz
* cd thrift-0.6.0
* ./configure --prefix=/usr/local/thrift --with-boost=/usr/local/boost_1_45_0
* make
* make install
* thrift now in /usr/local/thrift
* export THRIFT_HOME=/usr/local/thrift

FB303 Build/Install
* cd /buildloc/thrift-0.6.0/contrib/fb303
* ./bootstrap.sh
* cd /home/ptrout/thrift-0.6.0/contrib/fb303/cpp/gen-cpp
* vi fb303_types.cpp
** get rid of all fb_status:: (per this page: http://permalink.gmane.org/gmane.comp.lib.thrift.user/1443)
* cd /buildloc/thrift-0.6.0/contrib/fb303
* make
* make install

Apache Maven 3.0.2 Install (Required to build the Hive pieces)
* cd /usr/local
* tar -xzvf /buildloc/apache-maven-3.0.2.tar.gz
* export MAVEN_HOME=/usr/local/apache-maven-3.0.2
* export PATH=/usr/local/apache-maven-3.0.2/bin:$PATH

Install Hive 0.7.0 CDH3B4
* cd /usr/local
* tar -xzvf /buildloc/hive-0.7.0-CDH3B4.tar.gz
* cd hive-0.7.0-CDH3B4/conf
* cp /usr/local/hive/conf/hive-site.xml .
* edit hive-site.xml
* /usr/local/hive-0.7.0-CDH3B4/bin/hive appears to work
* export HIVE_HOME=/usr/local/hive-0.7.0CDH3B4

This is not a production system, but a test cluster, and I'm building on a fresh machine since, ideally, I just want to copy the shared libraries and ODBC-related configuration files to my production systems.

Thanks, in advance, for your assistance.

-Paul Trout