You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Sebastien FLAESCH <sf...@4js.com> on 2013/04/26 13:33:44 UTC
Compiling Hive ODBC
Hi all,
Just started with an Hadoop / Hive POC to write a
So far, I have installed:
So far I have downloaded the Hadoop and Hive packages from:
http://hadoop.apache.org/releases.html
http://hive.apache.org/releases.html
I took:
http://hadoop.apache.org/releases.html#18+April%2C+2013%3A+Release+0.23.7+available
and
http://hive.apache.org/releases.html#11+January%2C+2013%3A+release+0.10.0+available
After setting the env, it seems that hive is working:
hive@orca:~$ $HIVE_HOME/bin/hive
...
hive> create table t1 ( k int, s string );
OK
hive> select count(*) from t1;
Total MapReduce jobs = 1
...
OK
0
Time taken: 7.634 seconds
No I want to connect through ODBC...
Following the instructions of this page:
https://cwiki.apache.org/confluence/display/Hive/HiveODBC
Section "Hive Client Build/Setup", I tried:
hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
hive@orca:~/hive-0.10.0$ ant compile-cpp -Dthrift.home=/home/hive/hive-0.10.0/odbc
Buildfile: build.xml does not exist!
Build failed
Then, from the sources directory src/odbc:
hive@orca:~/hive-0.10.0$ cd src/odbc/
hive@orca:~/hive-0.10.0/src/odbc$ ant compile-cpp -Dthrift.home=/home/hive/hive-0.10.0/odbc
Buildfile: /home/hive/hive-0.10.0/src/odbc/build.xml
BUILD FAILED
/home/hive/hive-0.10.0/src/odbc/build.xml:30: The following error occurred while executing this line:
/home/hive/hive-0.10.0/src/build-common.xml:117: Problem: failed to create task or type osfamily
Cause: The name is undefined.
Action: Check the spelling.
Action: Check that any custom tasks/types have been declared.
Action: Check that any <presetdef>/<macrodef> declarations have taken place.
Total time: 0 seconds
Can someone help or point me to an up to date documentation?
Thanks!
Seb
Re: Compiling Hive ODBC
Posted by Sebastien FLAESCH <sf...@4js.com>.
So... could finally compile Hive ODBC by patching the Thrift.h header...
I think there is some configure header issue with the HAVE_* constants
the are defined in the <THRIFT_HOME>/include/thrift/config.h header.
It will only include the required headers if HAVE_CONFIG_H is defined:
#define HAVE_CONFIG_H 1 <--------- added line
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
Or is there a way to pass this define to ant ?
Seb
On 04/30/2013 04:39 PM, Sebastien FLAESCH wrote:
> FYI:
>
> hive@orca:~/hive-0.10.0/src$ g++ --version
> g++ (Debian 4.4.5-8) 4.4.5
>
> Seb
>
> On 04/30/2013 04:13 PM, Sebastien FLAESCH wrote:
>> Making some progress...
>>
>> After disabling some options to build thrift - because it installs some
>> files in the system directory (/usr/lib/php), and I - do not - want that,
>> I did following configure for thrift:
>>
>> ./configure --prefix=/home/hive/thrift-0.9.0 \
>> --with-qt4=no \
>> --with-csharp=no \
>> --with-erlang=no \
>> --with-python=no \
>> --with-perl=no \
>> --with-php=no \
>> --with-php_extension=no \
>> --with-ruby=no \
>> --with-haskell=no \
>> --with-go=no \
>> --with-d=no
>>
>> Then I could build thrift and fb303, but not when I try to build Hive
>> ODBC:
>>
>> hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
>> ...
>> ...
>> ...
>> [exec] g++ -Wall -g -fPIC -m32 -DARCH32
>> -I/home/hive/thrift-0.9.0/include/thrift
>> -I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
>> -I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp
>> -I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
>> -I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp
>> -I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
>>
>> -o /home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
>> [exec] In file included from
>> /usr/local/include/thrift/protocol/TProtocol.h:23,
>> [exec] from /usr/local/include/thrift/TProcessor.h:24,
>> [exec] from /usr/local/include/thrift/TDispatchProcessor.h:22,
>> [exec] from
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10,
>>
>>
>> [exec] from
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>>
>>
>> [exec] /usr/local/include/thrift/transport/TTransport.h:34: error:
>> expected constructor, destructor, or type conversion before ‘readAll’
>> [exec] /usr/local/include/thrift/transport/TTransport.h:107: error:
>> ‘uint32_t’ does not name a type
>> [exec] /usr/local/include/thrift/transport/TTransport.h:111: error:
>> ‘uint32_t’ does not name a type
>> [exec] /usr/local/include/thrift/transport/TTransport.h:124: error:
>> ‘uint32_t’ does not name a type
>> [exec] /usr/local/include/thrift/transport/TTransport.h:128: error:
>> ‘uint32_t’ does not name a type
>> [exec] /usr/local/include/thrift/transport/TTransport.h:139: error:
>> ‘uint32_t’ does not name a type
>>
>>
>> Any idea?
>>
>> Thanks
>> Seb
>>
>> On 04/30/2013 11:00 AM, Sebastien FLAESCH wrote:
>>> Thank you Carl, but I still have problems to build Hive ODBC:
>>>
>>> I followed the instruction from this page (I believe the doc is wrong):
>>>
>>> https://cwiki.apache.org/Hive/hiveodbc.html
>>>
>>> Where it says:
>>>
>>> "Build the Hive client by running the following command from
>>> HIVE_HOME..."
>>>
>>> Here is my env:
>>>
>>> ----------------------------------------------------------
>>> hive@orca:~$ cat env-apache.sh
>>> JAVA_HOME="/usr/lib/jvm/java-6-sun"
>>> export JAVA_HOME
>>> JRE_HOME="$JAVA_HOME/jre"
>>> export JRE_HOME
>>> PATH=$JAVA_HOME/bin:$PATH
>>>
>>> HADOOP_HOME=/home/hive/hadoop-0.23.7
>>> export HADOOP_HOME
>>> PATH=$HADOOP_HOME/bin:$PATH
>>> export PATH
>>>
>>> HIVE_HOME=/home/hive/hive-0.10.0
>>> export HIVE_HOME
>>> PATH=$HIVE_HOME/bin:$PATH
>>> export PATH
>>>
>>> THRIFT_HOME=/home/hive/thrift-0.9.0
>>> export THRIFT_HOME
>>> ----------------------------------------------------------
>>>
>>> But:
>>>
>>> ----------------------------------------------------------
>>> hive@orca:~$ cd $HIVE_HOME
>>> hive@orca:~/hive-0.10.0$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
>>> Buildfile: build.xml does not exist!
>>> Build failed
>>> ----------------------------------------------------------
>>>
>>>
>>> And when I go to the src directory, it goes further, but then
>>> I have missing Thrift header files:
>>>
>>> ----------------------------------------------------------
>>> hive@orca:~/hive-0.10.0$ cd src
>>> hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
>>> ...
>>> compile-cpp:
>>> [echo] Project: odbc
>>> [exec] mkdir -p /home/hive/hive-0.10.0/src/build/metastore/objs
>>> [exec] g++ -Wall -g -fPIC -m32 -DARCH32
>>> -I/home/hive/thrift-0.9.0/include/thrift
>>> -I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
>>> -I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp
>>> -I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
>>> -I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp
>>> -I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
>>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
>>>
>>>
>>> -o /home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
>>> [exec] In file included from
>>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>>>
>>>
>>>
>>> [exec]
>>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10:39:
>>>
>>>
>>> error: thrift/TDispatchProcessor.h: No such file or directory
>>> [exec] In file included from
>>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:11,
>>>
>>>
>>>
>>> [exec] from
>>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>>>
>>>
>>>
>>> [exec]
>>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:10:27:
>>>
>>>
>>> error: thrift/Thrift.h: No such file or directory
>>> [exec]
>>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:11:42:
>>>
>>>
>>> error: thrift/TApplicationException.h: No such file or directory
>>> ...
>>> ----------------------------------------------------------
>>>
>>> In fact there is no such directory:
>>>
>>> /home/hive/thrift-0.9.0/include/thrift/fb303
>>>
>>> I see only:
>>>
>>> ----------------------------------------------------------
>>> hive@orca:~/hive-0.10.0/src$ ls /home/hive/thrift-0.9.0/include/thrift
>>> c_glib
>>> ----------------------------------------------------------
>>>
>>> According to:
>>>
>>> http://mail-archives.apache.org/mod_mbox/hive-user/201110.mbox/%3C556325346CA26341B6F0530E07F90D960169F0FD464D@GBGH-EXCH-CMS.sig.ads%3E
>>>
>>>
>>>
>>>
>>> I need to build fb303 component of thrift...
>>>
>>> So I tried to build "fb303" in the thrift sources.
>>>
>>> Note that I ran thrift's configure with a non-system installation target
>>> directory:
>>>
>>> ----------------------------------------------------------
>>> hive@orca:~$ cd tmp/thrift-0.9.0/contrib/fb303
>>> hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ ./configure
>>> --prefix=/home/hive/thrift-0.9.0
>>> ----------------------------------------------------------
>>>
>>> But it could not find /usr/local/bin/thrift and
>>> /usr/local/include/thrift...
>>>
>>> So I created links:
>>>
>>> ----------------------------------------------------------
>>> /usr/local/bin/thrift -> /home/hive/thrift-0.9.0/bin/thrift
>>> /usr/local/include/thrift -> /home/hive/thrift-0.9.0/include/thrift
>>> ----------------------------------------------------------
>>>
>>> But:
>>>
>>> ----------------------------------------------------------
>>> hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ make
>>>
>>> make all-recursive
>>> make[1]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
>>> Making all in .
>>> make[2]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
>>> make[2]: Nothing to be done for `all-am'.
>>> make[2]: Leaving directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
>>> Making all in cpp
>>> make[2]: Entering directory
>>> `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
>>> make all-am
>>> make[3]: Entering directory
>>> `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
>>> g++ -DPACKAGE_NAME=\"libfb303\" -DPACKAGE_TARNAME=\"libfb303\"
>>> -DPACKAGE_VERSION=\"20080209\" -DPACKAGE_STRING=\"libfb303\ 20080209\"
>>> -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DHAVE_BOOST=/\*\*/
>>> -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1
>>> -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1
>>> -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1
>>> -DHAVE_INTTYPES_H=1 -DHAVE_NETINET_IN_H=1 -I. -I.. -Igen-cpp
>>> -I/usr/local/include/thrift -I/usr/include -Wall -O3 -MT
>>> FacebookService.o -MD -MP -MF .deps/FacebookService.Tpo -c -o
>>> FacebookService.o `test -f 'gen-cpp/FacebookService.cpp' || echo
>>> './'`gen-cpp/FacebookService.cpp
>>> In file included from gen-cpp/FacebookService.cpp:7:
>>> gen-cpp/FacebookService.h:10:39: error: thrift/TDispatchProcessor.h: No
>>> such file or directory
>>> In file included from gen-cpp/FacebookService.h:11,
>>> from gen-cpp/FacebookService.cpp:7:
>>>
>>> ----------------------------------------------------------
>>>
>>> ...
>>>
>>> Seb
>>>
>>>
>>> On 04/29/2013 08:48 PM, Carl Steinbach wrote:
>>>> Hi Sebastien,
>>>>
>>>> Most (if not all) of the ant targets will fail if run from the
>>>> submodule
>>>> directories. You must run the targets from the root source directory
>>>> instead.
>>>>
>>>> Thanks.
>>>>
>>>> Carl
>>>>
>>>>
>>>> On Fri, Apr 26, 2013 at 6:09 AM, Sebastien FLAESCH <sf@4js.com
>>>> <ma...@4js.com>> wrote:
>>>>
>>>> I have now installed Thrift, but I still get the error with the
>>>> "osfamily" task:
>>>>
>>>> Please help!
>>>>
>>>> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
>>>> -Dthrift.home=/home/hive/__thrift-0.9.0
>>>>
>>>>
>>>> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>>>>
>>>> BUILD FAILED
>>>> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The following error
>>>> occurred while executing this line:
>>>> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem: failed
>>>> to create task or type osfamily
>>>> Cause: The name is undefined.
>>>> Action: Check the spelling.
>>>> Action: Check that any custom tasks/types have been declared.
>>>> Action: Check that any <presetdef>/<macrodef> declarations have
>>>> taken place.
>>>>
>>>>
>>>> Total time: 0 seconds
>>>>
>>>> Seb
>>>>
>>>>
>>>> On 04/26/2013 01:40 PM, Sebastien FLAESCH wrote:
>>>>
>>>> Go it, need to install Apache Thrift first ... sorry for this mail.
>>>> Seb
>>>>
>>>> On 04/26/2013 01:33 PM, Sebastien FLAESCH wrote:
>>>>
>>>> Hi all,
>>>>
>>>> Just started with an Hadoop / Hive POC to write a
>>>>
>>>> So far, I have installed:
>>>>
>>>> So far I have downloaded the Hadoop and Hive packages from:
>>>>
>>>> http://hadoop.apache.org/__releases.html
>>>> <http://hadoop.apache.org/releases.html>
>>>> http://hive.apache.org/__releases.html
>>>> <http://hive.apache.org/releases.html>
>>>>
>>>> I took:
>>>>
>>>> http://hadoop.apache.org/__releases.html#18+April%2C+__2013%3A+Release+0.23.7+__available
>>>>
>>>>
>>>>
>>>> <http://hadoop.apache.org/releases.html#18+April%2C+2013%3A+Release+0.23.7+available>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> and
>>>>
>>>> http://hive.apache.org/__releases.html#11+January%2C+__2013%3A+release+0.10.0+__available
>>>>
>>>>
>>>>
>>>> <http://hive.apache.org/releases.html#11+January%2C+2013%3A+release+0.10.0+available>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> After setting the env, it seems that hive is working:
>>>>
>>>> hive@orca:~$ $HIVE_HOME/bin/hive
>>>> ...
>>>> hive> create table t1 ( k int, s string );
>>>> OK
>>>>
>>>> hive> select count(*) from t1;
>>>> Total MapReduce jobs = 1
>>>> ...
>>>> OK
>>>> 0
>>>> Time taken: 7.634 seconds
>>>>
>>>>
>>>> No I want to connect through ODBC...
>>>>
>>>> Following the instructions of this page:
>>>>
>>>> https://cwiki.apache.org/__confluence/display/Hive/__HiveODBC
>>>> <https://cwiki.apache.org/confluence/display/Hive/HiveODBC>
>>>>
>>>> Section "Hive Client Build/Setup", I tried:
>>>>
>>>> hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
>>>> hive@orca:~/hive-0.10.0$ ant compile-cpp
>>>> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
>>>> Buildfile: build.xml does not exist!
>>>> Build failed
>>>>
>>>> Then, from the sources directory src/odbc:
>>>>
>>>> hive@orca:~/hive-0.10.0$ cd src/odbc/
>>>> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
>>>> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
>>>> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>>>>
>>>> BUILD FAILED
>>>> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The
>>>> following error
>>>> occurred while executing this line:
>>>> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem:
>>>> failed to
>>>> create task or type osfamily
>>>> Cause: The name is undefined.
>>>> Action: Check the spelling.
>>>> Action: Check that any custom tasks/types have been declared.
>>>> Action: Check that any <presetdef>/<macrodef> declarations
>>>> have taken
>>>> place.
>>>>
>>>>
>>>> Total time: 0 seconds
>>>>
>>>>
>>>> Can someone help or point me to an up to date documentation?
>>>>
>>>> Thanks!
>>>> Seb
>>>>
>>>>
>>>>
>>>>
>>>
>>
>
Re: Compiling Hive ODBC
Posted by Sebastien FLAESCH <sf...@4js.com>.
FYI:
hive@orca:~/hive-0.10.0/src$ g++ --version
g++ (Debian 4.4.5-8) 4.4.5
Seb
On 04/30/2013 04:13 PM, Sebastien FLAESCH wrote:
> Making some progress...
>
> After disabling some options to build thrift - because it installs some
> files in the system directory (/usr/lib/php), and I - do not - want that,
> I did following configure for thrift:
>
> ./configure --prefix=/home/hive/thrift-0.9.0 \
> --with-qt4=no \
> --with-csharp=no \
> --with-erlang=no \
> --with-python=no \
> --with-perl=no \
> --with-php=no \
> --with-php_extension=no \
> --with-ruby=no \
> --with-haskell=no \
> --with-go=no \
> --with-d=no
>
> Then I could build thrift and fb303, but not when I try to build Hive ODBC:
>
> hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
> ...
> ...
> ...
> [exec] g++ -Wall -g -fPIC -m32 -DARCH32
> -I/home/hive/thrift-0.9.0/include/thrift
> -I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
> -I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
> -o /home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
> [exec] In file included from
> /usr/local/include/thrift/protocol/TProtocol.h:23,
> [exec] from /usr/local/include/thrift/TProcessor.h:24,
> [exec] from /usr/local/include/thrift/TDispatchProcessor.h:22,
> [exec] from
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10,
>
> [exec] from
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>
> [exec] /usr/local/include/thrift/transport/TTransport.h:34: error:
> expected constructor, destructor, or type conversion before ‘readAll’
> [exec] /usr/local/include/thrift/transport/TTransport.h:107: error:
> ‘uint32_t’ does not name a type
> [exec] /usr/local/include/thrift/transport/TTransport.h:111: error:
> ‘uint32_t’ does not name a type
> [exec] /usr/local/include/thrift/transport/TTransport.h:124: error:
> ‘uint32_t’ does not name a type
> [exec] /usr/local/include/thrift/transport/TTransport.h:128: error:
> ‘uint32_t’ does not name a type
> [exec] /usr/local/include/thrift/transport/TTransport.h:139: error:
> ‘uint32_t’ does not name a type
>
>
> Any idea?
>
> Thanks
> Seb
>
> On 04/30/2013 11:00 AM, Sebastien FLAESCH wrote:
>> Thank you Carl, but I still have problems to build Hive ODBC:
>>
>> I followed the instruction from this page (I believe the doc is wrong):
>>
>> https://cwiki.apache.org/Hive/hiveodbc.html
>>
>> Where it says:
>>
>> "Build the Hive client by running the following command from
>> HIVE_HOME..."
>>
>> Here is my env:
>>
>> ----------------------------------------------------------
>> hive@orca:~$ cat env-apache.sh
>> JAVA_HOME="/usr/lib/jvm/java-6-sun"
>> export JAVA_HOME
>> JRE_HOME="$JAVA_HOME/jre"
>> export JRE_HOME
>> PATH=$JAVA_HOME/bin:$PATH
>>
>> HADOOP_HOME=/home/hive/hadoop-0.23.7
>> export HADOOP_HOME
>> PATH=$HADOOP_HOME/bin:$PATH
>> export PATH
>>
>> HIVE_HOME=/home/hive/hive-0.10.0
>> export HIVE_HOME
>> PATH=$HIVE_HOME/bin:$PATH
>> export PATH
>>
>> THRIFT_HOME=/home/hive/thrift-0.9.0
>> export THRIFT_HOME
>> ----------------------------------------------------------
>>
>> But:
>>
>> ----------------------------------------------------------
>> hive@orca:~$ cd $HIVE_HOME
>> hive@orca:~/hive-0.10.0$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
>> Buildfile: build.xml does not exist!
>> Build failed
>> ----------------------------------------------------------
>>
>>
>> And when I go to the src directory, it goes further, but then
>> I have missing Thrift header files:
>>
>> ----------------------------------------------------------
>> hive@orca:~/hive-0.10.0$ cd src
>> hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
>> ...
>> compile-cpp:
>> [echo] Project: odbc
>> [exec] mkdir -p /home/hive/hive-0.10.0/src/build/metastore/objs
>> [exec] g++ -Wall -g -fPIC -m32 -DARCH32
>> -I/home/hive/thrift-0.9.0/include/thrift
>> -I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
>> -I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp
>> -I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
>> -I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp
>> -I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
>>
>> -o /home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
>> [exec] In file included from
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>>
>>
>> [exec]
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10:39:
>>
>> error: thrift/TDispatchProcessor.h: No such file or directory
>> [exec] In file included from
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:11,
>>
>>
>> [exec] from
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>>
>>
>> [exec]
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:10:27:
>>
>> error: thrift/Thrift.h: No such file or directory
>> [exec]
>> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:11:42:
>>
>> error: thrift/TApplicationException.h: No such file or directory
>> ...
>> ----------------------------------------------------------
>>
>> In fact there is no such directory:
>>
>> /home/hive/thrift-0.9.0/include/thrift/fb303
>>
>> I see only:
>>
>> ----------------------------------------------------------
>> hive@orca:~/hive-0.10.0/src$ ls /home/hive/thrift-0.9.0/include/thrift
>> c_glib
>> ----------------------------------------------------------
>>
>> According to:
>>
>> http://mail-archives.apache.org/mod_mbox/hive-user/201110.mbox/%3C556325346CA26341B6F0530E07F90D960169F0FD464D@GBGH-EXCH-CMS.sig.ads%3E
>>
>>
>>
>> I need to build fb303 component of thrift...
>>
>> So I tried to build "fb303" in the thrift sources.
>>
>> Note that I ran thrift's configure with a non-system installation target
>> directory:
>>
>> ----------------------------------------------------------
>> hive@orca:~$ cd tmp/thrift-0.9.0/contrib/fb303
>> hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ ./configure
>> --prefix=/home/hive/thrift-0.9.0
>> ----------------------------------------------------------
>>
>> But it could not find /usr/local/bin/thrift and
>> /usr/local/include/thrift...
>>
>> So I created links:
>>
>> ----------------------------------------------------------
>> /usr/local/bin/thrift -> /home/hive/thrift-0.9.0/bin/thrift
>> /usr/local/include/thrift -> /home/hive/thrift-0.9.0/include/thrift
>> ----------------------------------------------------------
>>
>> But:
>>
>> ----------------------------------------------------------
>> hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ make
>>
>> make all-recursive
>> make[1]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
>> Making all in .
>> make[2]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
>> make[2]: Nothing to be done for `all-am'.
>> make[2]: Leaving directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
>> Making all in cpp
>> make[2]: Entering directory
>> `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
>> make all-am
>> make[3]: Entering directory
>> `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
>> g++ -DPACKAGE_NAME=\"libfb303\" -DPACKAGE_TARNAME=\"libfb303\"
>> -DPACKAGE_VERSION=\"20080209\" -DPACKAGE_STRING=\"libfb303\ 20080209\"
>> -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DHAVE_BOOST=/\*\*/
>> -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1
>> -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1
>> -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1
>> -DHAVE_INTTYPES_H=1 -DHAVE_NETINET_IN_H=1 -I. -I.. -Igen-cpp
>> -I/usr/local/include/thrift -I/usr/include -Wall -O3 -MT
>> FacebookService.o -MD -MP -MF .deps/FacebookService.Tpo -c -o
>> FacebookService.o `test -f 'gen-cpp/FacebookService.cpp' || echo
>> './'`gen-cpp/FacebookService.cpp
>> In file included from gen-cpp/FacebookService.cpp:7:
>> gen-cpp/FacebookService.h:10:39: error: thrift/TDispatchProcessor.h: No
>> such file or directory
>> In file included from gen-cpp/FacebookService.h:11,
>> from gen-cpp/FacebookService.cpp:7:
>>
>> ----------------------------------------------------------
>>
>> ...
>>
>> Seb
>>
>>
>> On 04/29/2013 08:48 PM, Carl Steinbach wrote:
>>> Hi Sebastien,
>>>
>>> Most (if not all) of the ant targets will fail if run from the submodule
>>> directories. You must run the targets from the root source directory
>>> instead.
>>>
>>> Thanks.
>>>
>>> Carl
>>>
>>>
>>> On Fri, Apr 26, 2013 at 6:09 AM, Sebastien FLAESCH <sf@4js.com
>>> <ma...@4js.com>> wrote:
>>>
>>> I have now installed Thrift, but I still get the error with the
>>> "osfamily" task:
>>>
>>> Please help!
>>>
>>> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
>>> -Dthrift.home=/home/hive/__thrift-0.9.0
>>>
>>>
>>> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>>>
>>> BUILD FAILED
>>> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The following error
>>> occurred while executing this line:
>>> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem: failed
>>> to create task or type osfamily
>>> Cause: The name is undefined.
>>> Action: Check the spelling.
>>> Action: Check that any custom tasks/types have been declared.
>>> Action: Check that any <presetdef>/<macrodef> declarations have
>>> taken place.
>>>
>>>
>>> Total time: 0 seconds
>>>
>>> Seb
>>>
>>>
>>> On 04/26/2013 01:40 PM, Sebastien FLAESCH wrote:
>>>
>>> Go it, need to install Apache Thrift first ... sorry for this mail.
>>> Seb
>>>
>>> On 04/26/2013 01:33 PM, Sebastien FLAESCH wrote:
>>>
>>> Hi all,
>>>
>>> Just started with an Hadoop / Hive POC to write a
>>>
>>> So far, I have installed:
>>>
>>> So far I have downloaded the Hadoop and Hive packages from:
>>>
>>> http://hadoop.apache.org/__releases.html
>>> <http://hadoop.apache.org/releases.html>
>>> http://hive.apache.org/__releases.html
>>> <http://hive.apache.org/releases.html>
>>>
>>> I took:
>>>
>>> http://hadoop.apache.org/__releases.html#18+April%2C+__2013%3A+Release+0.23.7+__available
>>>
>>>
>>> <http://hadoop.apache.org/releases.html#18+April%2C+2013%3A+Release+0.23.7+available>
>>>
>>>
>>>
>>>
>>>
>>> and
>>>
>>> http://hive.apache.org/__releases.html#11+January%2C+__2013%3A+release+0.10.0+__available
>>>
>>>
>>> <http://hive.apache.org/releases.html#11+January%2C+2013%3A+release+0.10.0+available>
>>>
>>>
>>>
>>>
>>>
>>>
>>> After setting the env, it seems that hive is working:
>>>
>>> hive@orca:~$ $HIVE_HOME/bin/hive
>>> ...
>>> hive> create table t1 ( k int, s string );
>>> OK
>>>
>>> hive> select count(*) from t1;
>>> Total MapReduce jobs = 1
>>> ...
>>> OK
>>> 0
>>> Time taken: 7.634 seconds
>>>
>>>
>>> No I want to connect through ODBC...
>>>
>>> Following the instructions of this page:
>>>
>>> https://cwiki.apache.org/__confluence/display/Hive/__HiveODBC
>>> <https://cwiki.apache.org/confluence/display/Hive/HiveODBC>
>>>
>>> Section "Hive Client Build/Setup", I tried:
>>>
>>> hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
>>> hive@orca:~/hive-0.10.0$ ant compile-cpp
>>> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
>>> Buildfile: build.xml does not exist!
>>> Build failed
>>>
>>> Then, from the sources directory src/odbc:
>>>
>>> hive@orca:~/hive-0.10.0$ cd src/odbc/
>>> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
>>> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
>>> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>>>
>>> BUILD FAILED
>>> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The
>>> following error
>>> occurred while executing this line:
>>> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem:
>>> failed to
>>> create task or type osfamily
>>> Cause: The name is undefined.
>>> Action: Check the spelling.
>>> Action: Check that any custom tasks/types have been declared.
>>> Action: Check that any <presetdef>/<macrodef> declarations
>>> have taken
>>> place.
>>>
>>>
>>> Total time: 0 seconds
>>>
>>>
>>> Can someone help or point me to an up to date documentation?
>>>
>>> Thanks!
>>> Seb
>>>
>>>
>>>
>>>
>>
>
Re: Compiling Hive ODBC
Posted by Sebastien FLAESCH <sf...@4js.com>.
Making some progress...
After disabling some options to build thrift - because it installs some
files in the system directory (/usr/lib/php), and I - do not - want that,
I did following configure for thrift:
./configure --prefix=/home/hive/thrift-0.9.0 \
--with-qt4=no \
--with-csharp=no \
--with-erlang=no \
--with-python=no \
--with-perl=no \
--with-php=no \
--with-php_extension=no \
--with-ruby=no \
--with-haskell=no \
--with-go=no \
--with-d=no
Then I could build thrift and fb303, but not when I try to build Hive ODBC:
hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
...
...
...
[exec] g++ -Wall -g -fPIC -m32 -DARCH32 -I/home/hive/thrift-0.9.0/include/thrift -I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
-I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp -I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
-I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp -I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp -o
/home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
[exec] In file included from /usr/local/include/thrift/protocol/TProtocol.h:23,
[exec] from /usr/local/include/thrift/TProcessor.h:24,
[exec] from /usr/local/include/thrift/TDispatchProcessor.h:22,
[exec] from /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10,
[exec] from /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
[exec] /usr/local/include/thrift/transport/TTransport.h:34: error: expected constructor, destructor, or type conversion before ‘readAll’
[exec] /usr/local/include/thrift/transport/TTransport.h:107: error: ‘uint32_t’ does not name a type
[exec] /usr/local/include/thrift/transport/TTransport.h:111: error: ‘uint32_t’ does not name a type
[exec] /usr/local/include/thrift/transport/TTransport.h:124: error: ‘uint32_t’ does not name a type
[exec] /usr/local/include/thrift/transport/TTransport.h:128: error: ‘uint32_t’ does not name a type
[exec] /usr/local/include/thrift/transport/TTransport.h:139: error: ‘uint32_t’ does not name a type
Any idea?
Thanks
Seb
On 04/30/2013 11:00 AM, Sebastien FLAESCH wrote:
> Thank you Carl, but I still have problems to build Hive ODBC:
>
> I followed the instruction from this page (I believe the doc is wrong):
>
> https://cwiki.apache.org/Hive/hiveodbc.html
>
> Where it says:
>
> "Build the Hive client by running the following command from HIVE_HOME..."
>
> Here is my env:
>
> ----------------------------------------------------------
> hive@orca:~$ cat env-apache.sh
> JAVA_HOME="/usr/lib/jvm/java-6-sun"
> export JAVA_HOME
> JRE_HOME="$JAVA_HOME/jre"
> export JRE_HOME
> PATH=$JAVA_HOME/bin:$PATH
>
> HADOOP_HOME=/home/hive/hadoop-0.23.7
> export HADOOP_HOME
> PATH=$HADOOP_HOME/bin:$PATH
> export PATH
>
> HIVE_HOME=/home/hive/hive-0.10.0
> export HIVE_HOME
> PATH=$HIVE_HOME/bin:$PATH
> export PATH
>
> THRIFT_HOME=/home/hive/thrift-0.9.0
> export THRIFT_HOME
> ----------------------------------------------------------
>
> But:
>
> ----------------------------------------------------------
> hive@orca:~$ cd $HIVE_HOME
> hive@orca:~/hive-0.10.0$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
> Buildfile: build.xml does not exist!
> Build failed
> ----------------------------------------------------------
>
>
> And when I go to the src directory, it goes further, but then
> I have missing Thrift header files:
>
> ----------------------------------------------------------
> hive@orca:~/hive-0.10.0$ cd src
> hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
> ...
> compile-cpp:
> [echo] Project: odbc
> [exec] mkdir -p /home/hive/hive-0.10.0/src/build/metastore/objs
> [exec] g++ -Wall -g -fPIC -m32 -DARCH32
> -I/home/hive/thrift-0.9.0/include/thrift
> -I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
> -I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp
> -I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
> -o /home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
> [exec] In file included from
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>
> [exec]
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10:39:
> error: thrift/TDispatchProcessor.h: No such file or directory
> [exec] In file included from
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:11,
>
> [exec] from
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
>
> [exec]
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:10:27:
> error: thrift/Thrift.h: No such file or directory
> [exec]
> /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:11:42:
> error: thrift/TApplicationException.h: No such file or directory
> ...
> ----------------------------------------------------------
>
> In fact there is no such directory:
>
> /home/hive/thrift-0.9.0/include/thrift/fb303
>
> I see only:
>
> ----------------------------------------------------------
> hive@orca:~/hive-0.10.0/src$ ls /home/hive/thrift-0.9.0/include/thrift
> c_glib
> ----------------------------------------------------------
>
> According to:
>
> http://mail-archives.apache.org/mod_mbox/hive-user/201110.mbox/%3C556325346CA26341B6F0530E07F90D960169F0FD464D@GBGH-EXCH-CMS.sig.ads%3E
>
>
> I need to build fb303 component of thrift...
>
> So I tried to build "fb303" in the thrift sources.
>
> Note that I ran thrift's configure with a non-system installation target
> directory:
>
> ----------------------------------------------------------
> hive@orca:~$ cd tmp/thrift-0.9.0/contrib/fb303
> hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ ./configure
> --prefix=/home/hive/thrift-0.9.0
> ----------------------------------------------------------
>
> But it could not find /usr/local/bin/thrift and
> /usr/local/include/thrift...
>
> So I created links:
>
> ----------------------------------------------------------
> /usr/local/bin/thrift -> /home/hive/thrift-0.9.0/bin/thrift
> /usr/local/include/thrift -> /home/hive/thrift-0.9.0/include/thrift
> ----------------------------------------------------------
>
> But:
>
> ----------------------------------------------------------
> hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ make
>
> make all-recursive
> make[1]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
> Making all in .
> make[2]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
> make[2]: Nothing to be done for `all-am'.
> make[2]: Leaving directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
> Making all in cpp
> make[2]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
> make all-am
> make[3]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
> g++ -DPACKAGE_NAME=\"libfb303\" -DPACKAGE_TARNAME=\"libfb303\"
> -DPACKAGE_VERSION=\"20080209\" -DPACKAGE_STRING=\"libfb303\ 20080209\"
> -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DHAVE_BOOST=/\*\*/
> -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1
> -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1
> -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1
> -DHAVE_INTTYPES_H=1 -DHAVE_NETINET_IN_H=1 -I. -I.. -Igen-cpp
> -I/usr/local/include/thrift -I/usr/include -Wall -O3 -MT
> FacebookService.o -MD -MP -MF .deps/FacebookService.Tpo -c -o
> FacebookService.o `test -f 'gen-cpp/FacebookService.cpp' || echo
> './'`gen-cpp/FacebookService.cpp
> In file included from gen-cpp/FacebookService.cpp:7:
> gen-cpp/FacebookService.h:10:39: error: thrift/TDispatchProcessor.h: No
> such file or directory
> In file included from gen-cpp/FacebookService.h:11,
> from gen-cpp/FacebookService.cpp:7:
>
> ----------------------------------------------------------
>
> ...
>
> Seb
>
>
> On 04/29/2013 08:48 PM, Carl Steinbach wrote:
>> Hi Sebastien,
>>
>> Most (if not all) of the ant targets will fail if run from the submodule
>> directories. You must run the targets from the root source directory
>> instead.
>>
>> Thanks.
>>
>> Carl
>>
>>
>> On Fri, Apr 26, 2013 at 6:09 AM, Sebastien FLAESCH <sf@4js.com
>> <ma...@4js.com>> wrote:
>>
>> I have now installed Thrift, but I still get the error with the
>> "osfamily" task:
>>
>> Please help!
>>
>> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
>> -Dthrift.home=/home/hive/__thrift-0.9.0
>>
>>
>> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>>
>> BUILD FAILED
>> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The following error
>> occurred while executing this line:
>> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem: failed
>> to create task or type osfamily
>> Cause: The name is undefined.
>> Action: Check the spelling.
>> Action: Check that any custom tasks/types have been declared.
>> Action: Check that any <presetdef>/<macrodef> declarations have
>> taken place.
>>
>>
>> Total time: 0 seconds
>>
>> Seb
>>
>>
>> On 04/26/2013 01:40 PM, Sebastien FLAESCH wrote:
>>
>> Go it, need to install Apache Thrift first ... sorry for this mail.
>> Seb
>>
>> On 04/26/2013 01:33 PM, Sebastien FLAESCH wrote:
>>
>> Hi all,
>>
>> Just started with an Hadoop / Hive POC to write a
>>
>> So far, I have installed:
>>
>> So far I have downloaded the Hadoop and Hive packages from:
>>
>> http://hadoop.apache.org/__releases.html
>> <http://hadoop.apache.org/releases.html>
>> http://hive.apache.org/__releases.html
>> <http://hive.apache.org/releases.html>
>>
>> I took:
>>
>> http://hadoop.apache.org/__releases.html#18+April%2C+__2013%3A+Release+0.23.7+__available
>>
>> <http://hadoop.apache.org/releases.html#18+April%2C+2013%3A+Release+0.23.7+available>
>>
>>
>>
>>
>> and
>>
>> http://hive.apache.org/__releases.html#11+January%2C+__2013%3A+release+0.10.0+__available
>>
>> <http://hive.apache.org/releases.html#11+January%2C+2013%3A+release+0.10.0+available>
>>
>>
>>
>>
>>
>> After setting the env, it seems that hive is working:
>>
>> hive@orca:~$ $HIVE_HOME/bin/hive
>> ...
>> hive> create table t1 ( k int, s string );
>> OK
>>
>> hive> select count(*) from t1;
>> Total MapReduce jobs = 1
>> ...
>> OK
>> 0
>> Time taken: 7.634 seconds
>>
>>
>> No I want to connect through ODBC...
>>
>> Following the instructions of this page:
>>
>> https://cwiki.apache.org/__confluence/display/Hive/__HiveODBC
>> <https://cwiki.apache.org/confluence/display/Hive/HiveODBC>
>>
>> Section "Hive Client Build/Setup", I tried:
>>
>> hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
>> hive@orca:~/hive-0.10.0$ ant compile-cpp
>> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
>> Buildfile: build.xml does not exist!
>> Build failed
>>
>> Then, from the sources directory src/odbc:
>>
>> hive@orca:~/hive-0.10.0$ cd src/odbc/
>> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
>> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
>> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>>
>> BUILD FAILED
>> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The
>> following error
>> occurred while executing this line:
>> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem:
>> failed to
>> create task or type osfamily
>> Cause: The name is undefined.
>> Action: Check the spelling.
>> Action: Check that any custom tasks/types have been declared.
>> Action: Check that any <presetdef>/<macrodef> declarations
>> have taken
>> place.
>>
>>
>> Total time: 0 seconds
>>
>>
>> Can someone help or point me to an up to date documentation?
>>
>> Thanks!
>> Seb
>>
>>
>>
>>
>
Re: Compiling Hive ODBC
Posted by Sebastien FLAESCH <sf...@4js.com>.
Thank you Carl, but I still have problems to build Hive ODBC:
I followed the instruction from this page (I believe the doc is wrong):
https://cwiki.apache.org/Hive/hiveodbc.html
Where it says:
"Build the Hive client by running the following command from HIVE_HOME..."
Here is my env:
----------------------------------------------------------
hive@orca:~$ cat env-apache.sh
JAVA_HOME="/usr/lib/jvm/java-6-sun"
export JAVA_HOME
JRE_HOME="$JAVA_HOME/jre"
export JRE_HOME
PATH=$JAVA_HOME/bin:$PATH
HADOOP_HOME=/home/hive/hadoop-0.23.7
export HADOOP_HOME
PATH=$HADOOP_HOME/bin:$PATH
export PATH
HIVE_HOME=/home/hive/hive-0.10.0
export HIVE_HOME
PATH=$HIVE_HOME/bin:$PATH
export PATH
THRIFT_HOME=/home/hive/thrift-0.9.0
export THRIFT_HOME
----------------------------------------------------------
But:
----------------------------------------------------------
hive@orca:~$ cd $HIVE_HOME
hive@orca:~/hive-0.10.0$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
Buildfile: build.xml does not exist!
Build failed
----------------------------------------------------------
And when I go to the src directory, it goes further, but then
I have missing Thrift header files:
----------------------------------------------------------
hive@orca:~/hive-0.10.0$ cd src
hive@orca:~/hive-0.10.0/src$ ant compile-cpp -Dthrift.home=$THRIFT_HOME
...
compile-cpp:
[echo] Project: odbc
[exec] mkdir -p /home/hive/hive-0.10.0/src/build/metastore/objs
[exec] g++ -Wall -g -fPIC -m32 -DARCH32 -I/home/hive/thrift-0.9.0/include/thrift -I/home/hive/thrift-0.9.0/include/thrift/fb303 -I/include
-I/home/hive/hive-0.10.0/src/service/src/gen/thrift/gen-cpp -I/home/hive/hive-0.10.0/src/ql/src/gen/thrift/gen-cpp
-I/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp -I/home/hive/hive-0.10.0/src/odbc/src/cpp -c
/home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp -o
/home/hive/hive-0.10.0/src/build/metastore/objs/ThriftHiveMetastore.o
[exec] In file included from /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
[exec] /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:10:39: error: thrift/TDispatchProcessor.h: No such file
or directory
[exec] In file included from /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:11,
[exec] from /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:7:
[exec] /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:10:27: error: thrift/Thrift.h: No such file or directory
[exec] /home/hive/hive-0.10.0/src/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h:11:42: error: thrift/TApplicationException.h: No such
file or directory
...
----------------------------------------------------------
In fact there is no such directory:
/home/hive/thrift-0.9.0/include/thrift/fb303
I see only:
----------------------------------------------------------
hive@orca:~/hive-0.10.0/src$ ls /home/hive/thrift-0.9.0/include/thrift
c_glib
----------------------------------------------------------
According to:
http://mail-archives.apache.org/mod_mbox/hive-user/201110.mbox/%3C556325346CA26341B6F0530E07F90D960169F0FD464D@GBGH-EXCH-CMS.sig.ads%3E
I need to build fb303 component of thrift...
So I tried to build "fb303" in the thrift sources.
Note that I ran thrift's configure with a non-system installation target directory:
----------------------------------------------------------
hive@orca:~$ cd tmp/thrift-0.9.0/contrib/fb303
hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ ./configure --prefix=/home/hive/thrift-0.9.0
----------------------------------------------------------
But it could not find /usr/local/bin/thrift and /usr/local/include/thrift...
So I created links:
----------------------------------------------------------
/usr/local/bin/thrift -> /home/hive/thrift-0.9.0/bin/thrift
/usr/local/include/thrift -> /home/hive/thrift-0.9.0/include/thrift
----------------------------------------------------------
But:
----------------------------------------------------------
hive@orca:~/tmp/thrift-0.9.0/contrib/fb303$ make
make all-recursive
make[1]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
Making all in .
make[2]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
make[2]: Nothing to be done for `all-am'.
make[2]: Leaving directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303'
Making all in cpp
make[2]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
make all-am
make[3]: Entering directory `/home/hive/tmp/thrift-0.9.0/contrib/fb303/cpp'
g++ -DPACKAGE_NAME=\"libfb303\" -DPACKAGE_TARNAME=\"libfb303\" -DPACKAGE_VERSION=\"20080209\" -DPACKAGE_STRING=\"libfb303\ 20080209\"
-DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DHAVE_BOOST=/\*\*/ -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1
-DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_INTTYPES_H=1
-DHAVE_NETINET_IN_H=1 -I. -I.. -Igen-cpp -I/usr/local/include/thrift -I/usr/include -Wall -O3 -MT FacebookService.o -MD -MP -MF
.deps/FacebookService.Tpo -c -o FacebookService.o `test -f 'gen-cpp/FacebookService.cpp' || echo './'`gen-cpp/FacebookService.cpp
In file included from gen-cpp/FacebookService.cpp:7:
gen-cpp/FacebookService.h:10:39: error: thrift/TDispatchProcessor.h: No such file or directory
In file included from gen-cpp/FacebookService.h:11,
from gen-cpp/FacebookService.cpp:7:
----------------------------------------------------------
...
Seb
On 04/29/2013 08:48 PM, Carl Steinbach wrote:
> Hi Sebastien,
>
> Most (if not all) of the ant targets will fail if run from the submodule
> directories. You must run the targets from the root source directory
> instead.
>
> Thanks.
>
> Carl
>
>
> On Fri, Apr 26, 2013 at 6:09 AM, Sebastien FLAESCH <sf@4js.com
> <ma...@4js.com>> wrote:
>
> I have now installed Thrift, but I still get the error with the
> "osfamily" task:
>
> Please help!
>
> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
> -Dthrift.home=/home/hive/__thrift-0.9.0
>
>
> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>
> BUILD FAILED
> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The following error
> occurred while executing this line:
> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem: failed
> to create task or type osfamily
> Cause: The name is undefined.
> Action: Check the spelling.
> Action: Check that any custom tasks/types have been declared.
> Action: Check that any <presetdef>/<macrodef> declarations have
> taken place.
>
>
> Total time: 0 seconds
>
> Seb
>
>
> On 04/26/2013 01:40 PM, Sebastien FLAESCH wrote:
>
> Go it, need to install Apache Thrift first ... sorry for this mail.
> Seb
>
> On 04/26/2013 01:33 PM, Sebastien FLAESCH wrote:
>
> Hi all,
>
> Just started with an Hadoop / Hive POC to write a
>
> So far, I have installed:
>
> So far I have downloaded the Hadoop and Hive packages from:
>
> http://hadoop.apache.org/__releases.html
> <http://hadoop.apache.org/releases.html>
> http://hive.apache.org/__releases.html
> <http://hive.apache.org/releases.html>
>
> I took:
>
> http://hadoop.apache.org/__releases.html#18+April%2C+__2013%3A+Release+0.23.7+__available
> <http://hadoop.apache.org/releases.html#18+April%2C+2013%3A+Release+0.23.7+available>
>
>
>
> and
>
> http://hive.apache.org/__releases.html#11+January%2C+__2013%3A+release+0.10.0+__available
> <http://hive.apache.org/releases.html#11+January%2C+2013%3A+release+0.10.0+available>
>
>
>
>
> After setting the env, it seems that hive is working:
>
> hive@orca:~$ $HIVE_HOME/bin/hive
> ...
> hive> create table t1 ( k int, s string );
> OK
>
> hive> select count(*) from t1;
> Total MapReduce jobs = 1
> ...
> OK
> 0
> Time taken: 7.634 seconds
>
>
> No I want to connect through ODBC...
>
> Following the instructions of this page:
>
> https://cwiki.apache.org/__confluence/display/Hive/__HiveODBC <https://cwiki.apache.org/confluence/display/Hive/HiveODBC>
>
> Section "Hive Client Build/Setup", I tried:
>
> hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
> hive@orca:~/hive-0.10.0$ ant compile-cpp
> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
> Buildfile: build.xml does not exist!
> Build failed
>
> Then, from the sources directory src/odbc:
>
> hive@orca:~/hive-0.10.0$ cd src/odbc/
> hive@orca:~/hive-0.10.0/src/__odbc$ ant compile-cpp
> -Dthrift.home=/home/hive/hive-__0.10.0/odbc
> Buildfile: /home/hive/hive-0.10.0/src/__odbc/build.xml
>
> BUILD FAILED
> /home/hive/hive-0.10.0/src/__odbc/build.xml:30: The
> following error
> occurred while executing this line:
> /home/hive/hive-0.10.0/src/__build-common.xml:117: Problem:
> failed to
> create task or type osfamily
> Cause: The name is undefined.
> Action: Check the spelling.
> Action: Check that any custom tasks/types have been declared.
> Action: Check that any <presetdef>/<macrodef> declarations
> have taken
> place.
>
>
> Total time: 0 seconds
>
>
> Can someone help or point me to an up to date documentation?
>
> Thanks!
> Seb
>
>
>
>
Re: Compiling Hive ODBC
Posted by Carl Steinbach <cw...@gmail.com>.
Hi Sebastien,
Most (if not all) of the ant targets will fail if run from the submodule
directories. You must run the targets from the root source directory
instead.
Thanks.
Carl
On Fri, Apr 26, 2013 at 6:09 AM, Sebastien FLAESCH <sf...@4js.com> wrote:
> I have now installed Thrift, but I still get the error with the "osfamily"
> task:
>
> Please help!
>
> hive@orca:~/hive-0.10.0/src/**odbc$ ant compile-cpp
> -Dthrift.home=/home/hive/**thrift-0.9.0
>
>
> Buildfile: /home/hive/hive-0.10.0/src/**odbc/build.xml
>
> BUILD FAILED
> /home/hive/hive-0.10.0/src/**odbc/build.xml:30: The following error
> occurred while executing this line:
> /home/hive/hive-0.10.0/src/**build-common.xml:117: Problem: failed to
> create task or type osfamily
> Cause: The name is undefined.
> Action: Check the spelling.
> Action: Check that any custom tasks/types have been declared.
> Action: Check that any <presetdef>/<macrodef> declarations have taken
> place.
>
>
> Total time: 0 seconds
>
> Seb
>
>
> On 04/26/2013 01:40 PM, Sebastien FLAESCH wrote:
>
>> Go it, need to install Apache Thrift first ... sorry for this mail.
>> Seb
>>
>> On 04/26/2013 01:33 PM, Sebastien FLAESCH wrote:
>>
>>> Hi all,
>>>
>>> Just started with an Hadoop / Hive POC to write a
>>>
>>> So far, I have installed:
>>>
>>> So far I have downloaded the Hadoop and Hive packages from:
>>>
>>> http://hadoop.apache.org/**releases.html<http://hadoop.apache.org/releases.html>
>>> http://hive.apache.org/**releases.html<http://hive.apache.org/releases.html>
>>>
>>> I took:
>>>
>>> http://hadoop.apache.org/**releases.html#18+April%2C+**
>>> 2013%3A+Release+0.23.7+**available<http://hadoop.apache.org/releases.html#18+April%2C+2013%3A+Release+0.23.7+available>
>>>
>>>
>>>
>>> and
>>>
>>> http://hive.apache.org/**releases.html#11+January%2C+**
>>> 2013%3A+release+0.10.0+**available<http://hive.apache.org/releases.html#11+January%2C+2013%3A+release+0.10.0+available>
>>>
>>>
>>>
>>>
>>> After setting the env, it seems that hive is working:
>>>
>>> hive@orca:~$ $HIVE_HOME/bin/hive
>>> ...
>>> hive> create table t1 ( k int, s string );
>>> OK
>>>
>>> hive> select count(*) from t1;
>>> Total MapReduce jobs = 1
>>> ...
>>> OK
>>> 0
>>> Time taken: 7.634 seconds
>>>
>>>
>>> No I want to connect through ODBC...
>>>
>>> Following the instructions of this page:
>>>
>>> https://cwiki.apache.org/**confluence/display/Hive/**HiveODBC<https://cwiki.apache.org/confluence/display/Hive/HiveODBC>
>>>
>>> Section "Hive Client Build/Setup", I tried:
>>>
>>> hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
>>> hive@orca:~/hive-0.10.0$ ant compile-cpp
>>> -Dthrift.home=/home/hive/hive-**0.10.0/odbc
>>> Buildfile: build.xml does not exist!
>>> Build failed
>>>
>>> Then, from the sources directory src/odbc:
>>>
>>> hive@orca:~/hive-0.10.0$ cd src/odbc/
>>> hive@orca:~/hive-0.10.0/src/**odbc$ ant compile-cpp
>>> -Dthrift.home=/home/hive/hive-**0.10.0/odbc
>>> Buildfile: /home/hive/hive-0.10.0/src/**odbc/build.xml
>>>
>>> BUILD FAILED
>>> /home/hive/hive-0.10.0/src/**odbc/build.xml:30: The following error
>>> occurred while executing this line:
>>> /home/hive/hive-0.10.0/src/**build-common.xml:117: Problem: failed to
>>> create task or type osfamily
>>> Cause: The name is undefined.
>>> Action: Check the spelling.
>>> Action: Check that any custom tasks/types have been declared.
>>> Action: Check that any <presetdef>/<macrodef> declarations have taken
>>> place.
>>>
>>>
>>> Total time: 0 seconds
>>>
>>>
>>> Can someone help or point me to an up to date documentation?
>>>
>>> Thanks!
>>> Seb
>>>
>>>
>>
>
Re: Compiling Hive ODBC
Posted by Sebastien FLAESCH <sf...@4js.com>.
I have now installed Thrift, but I still get the error with the "osfamily" task:
Please help!
hive@orca:~/hive-0.10.0/src/odbc$ ant compile-cpp -Dthrift.home=/home/hive/thrift-0.9.0
Buildfile: /home/hive/hive-0.10.0/src/odbc/build.xml
BUILD FAILED
/home/hive/hive-0.10.0/src/odbc/build.xml:30: The following error occurred while executing this line:
/home/hive/hive-0.10.0/src/build-common.xml:117: Problem: failed to create task or type osfamily
Cause: The name is undefined.
Action: Check the spelling.
Action: Check that any custom tasks/types have been declared.
Action: Check that any <presetdef>/<macrodef> declarations have taken place.
Total time: 0 seconds
Seb
On 04/26/2013 01:40 PM, Sebastien FLAESCH wrote:
> Go it, need to install Apache Thrift first ... sorry for this mail.
> Seb
>
> On 04/26/2013 01:33 PM, Sebastien FLAESCH wrote:
>> Hi all,
>>
>> Just started with an Hadoop / Hive POC to write a
>>
>> So far, I have installed:
>>
>> So far I have downloaded the Hadoop and Hive packages from:
>>
>> http://hadoop.apache.org/releases.html
>> http://hive.apache.org/releases.html
>>
>> I took:
>>
>> http://hadoop.apache.org/releases.html#18+April%2C+2013%3A+Release+0.23.7+available
>>
>>
>>
>> and
>>
>> http://hive.apache.org/releases.html#11+January%2C+2013%3A+release+0.10.0+available
>>
>>
>>
>>
>> After setting the env, it seems that hive is working:
>>
>> hive@orca:~$ $HIVE_HOME/bin/hive
>> ...
>> hive> create table t1 ( k int, s string );
>> OK
>>
>> hive> select count(*) from t1;
>> Total MapReduce jobs = 1
>> ...
>> OK
>> 0
>> Time taken: 7.634 seconds
>>
>>
>> No I want to connect through ODBC...
>>
>> Following the instructions of this page:
>>
>> https://cwiki.apache.org/confluence/display/Hive/HiveODBC
>>
>> Section "Hive Client Build/Setup", I tried:
>>
>> hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
>> hive@orca:~/hive-0.10.0$ ant compile-cpp
>> -Dthrift.home=/home/hive/hive-0.10.0/odbc
>> Buildfile: build.xml does not exist!
>> Build failed
>>
>> Then, from the sources directory src/odbc:
>>
>> hive@orca:~/hive-0.10.0$ cd src/odbc/
>> hive@orca:~/hive-0.10.0/src/odbc$ ant compile-cpp
>> -Dthrift.home=/home/hive/hive-0.10.0/odbc
>> Buildfile: /home/hive/hive-0.10.0/src/odbc/build.xml
>>
>> BUILD FAILED
>> /home/hive/hive-0.10.0/src/odbc/build.xml:30: The following error
>> occurred while executing this line:
>> /home/hive/hive-0.10.0/src/build-common.xml:117: Problem: failed to
>> create task or type osfamily
>> Cause: The name is undefined.
>> Action: Check the spelling.
>> Action: Check that any custom tasks/types have been declared.
>> Action: Check that any <presetdef>/<macrodef> declarations have taken
>> place.
>>
>>
>> Total time: 0 seconds
>>
>>
>> Can someone help or point me to an up to date documentation?
>>
>> Thanks!
>> Seb
>>
>
Re: Compiling Hive ODBC
Posted by Sebastien FLAESCH <sf...@4js.com>.
Go it, need to install Apache Thrift first ... sorry for this mail.
Seb
On 04/26/2013 01:33 PM, Sebastien FLAESCH wrote:
> Hi all,
>
> Just started with an Hadoop / Hive POC to write a
>
> So far, I have installed:
>
> So far I have downloaded the Hadoop and Hive packages from:
>
> http://hadoop.apache.org/releases.html
> http://hive.apache.org/releases.html
>
> I took:
>
> http://hadoop.apache.org/releases.html#18+April%2C+2013%3A+Release+0.23.7+available
>
>
> and
>
> http://hive.apache.org/releases.html#11+January%2C+2013%3A+release+0.10.0+available
>
>
>
> After setting the env, it seems that hive is working:
>
> hive@orca:~$ $HIVE_HOME/bin/hive
> ...
> hive> create table t1 ( k int, s string );
> OK
>
> hive> select count(*) from t1;
> Total MapReduce jobs = 1
> ...
> OK
> 0
> Time taken: 7.634 seconds
>
>
> No I want to connect through ODBC...
>
> Following the instructions of this page:
>
> https://cwiki.apache.org/confluence/display/Hive/HiveODBC
>
> Section "Hive Client Build/Setup", I tried:
>
> hive@orca:~/hive-0.10.0$ cd $HIVE_HOME
> hive@orca:~/hive-0.10.0$ ant compile-cpp
> -Dthrift.home=/home/hive/hive-0.10.0/odbc
> Buildfile: build.xml does not exist!
> Build failed
>
> Then, from the sources directory src/odbc:
>
> hive@orca:~/hive-0.10.0$ cd src/odbc/
> hive@orca:~/hive-0.10.0/src/odbc$ ant compile-cpp
> -Dthrift.home=/home/hive/hive-0.10.0/odbc
> Buildfile: /home/hive/hive-0.10.0/src/odbc/build.xml
>
> BUILD FAILED
> /home/hive/hive-0.10.0/src/odbc/build.xml:30: The following error
> occurred while executing this line:
> /home/hive/hive-0.10.0/src/build-common.xml:117: Problem: failed to
> create task or type osfamily
> Cause: The name is undefined.
> Action: Check the spelling.
> Action: Check that any custom tasks/types have been declared.
> Action: Check that any <presetdef>/<macrodef> declarations have taken
> place.
>
>
> Total time: 0 seconds
>
>
> Can someone help or point me to an up to date documentation?
>
> Thanks!
> Seb
>