python libraries to execute or call hive queries

[Giri P <gp...@gmail.com>]

Hi All,

Can anyone suggest any python libraries to call hive queries from python
scripts ?


what is the best practice to execute queries from python like using hive
cli , beeline, jdbc etc..,

Thanks
Giri

RE: python libraries to execute or call hive queries

[rakesh sharma <ra...@hotmail.com>]

Hi Gopal
Have you tried pyhs2 libraryIt has many useful functions to retrieve the data
thanksrakesh

> Date: Fri, 28 Aug 2015 11:53:20 -0700
> Subject: Re: python libraries to execute or call hive queries
> From: gopalv@apache.org
> To: user@hive.apache.org
> 
> 
> > Can anyone suggest any python libraries to call hive queries from python
> >scripts ?
> 
> https://cwiki.apache.org/confluence/display/Hive/HiveClient#HiveClient-Pyth
> on
> 
> 
> Though I suspect that's out of date.
> 
> https://github.com/t3rmin4t0r/amplab-benchmark/blob/master/runner/run_query
> .py#L604
> 
> 
> is roughly the way to cut-paste that into working form (for hive-13),
> though you've got to use the exact thrift version of the HiveServer2 you
> run against.
> 
> Though, recently I've noticed the SQLAlchemy wrappers to be more
> convenient 
> 
> https://github.com/dropbox/PyHive/blob/master/pyhive/sqlalchemy_hive.py
> 
> 
> Irrespective of the method of access, the only consistent way to talk to
> Hive is over the JDBC interaction layer (Thrift server).
> 
> Launching bin/hive via Subprocess will work, but I've found that reading
> the results out with a regex has more parsing issues than I'd like.
> 
> Cheers,
> Gopal
> 
> 
 		 	   		  

Re: python libraries to execute or call hive queries

[Gopal Vijayaraghavan <go...@apache.org>]

> Can anyone suggest any python libraries to call hive queries from python
>scripts ?

https://cwiki.apache.org/confluence/display/Hive/HiveClient#HiveClient-Pyth
on


Though I suspect that's out of date.

https://github.com/t3rmin4t0r/amplab-benchmark/blob/master/runner/run_query
.py#L604


is roughly the way to cut-paste that into working form (for hive-13),
though you've got to use the exact thrift version of the HiveServer2 you
run against.

Though, recently I've noticed the SQLAlchemy wrappers to be more
convenient 

https://github.com/dropbox/PyHive/blob/master/pyhive/sqlalchemy_hive.py


Irrespective of the method of access, the only consistent way to talk to
Hive is over the JDBC interaction layer (Thrift server).

Launching bin/hive via Subprocess will work, but I've found that reading
the results out with a regex has more parsing issues than I'd like.

Cheers,
Gopal