You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Gary Richardson <ga...@gmail.com> on 2009/02/19 22:49:09 UTC

Is there a network interface?

Hi,

How do you execute hive queries programmatically and/or remotely? I'm still
new to hadoop and hive so I may be missing something obvious.

I recognize that a PDO/DBI/JDBC type connection doesn't make sense with
Hive. Nor does running queries from a web request.

I'd like to do something like:

$queryid = ExecuteQuery($query);

while(!CheckQueryStatus($queryid)) {
  sleep(60);
}

$data = RetreiveResultsFromHDFS();

And I'd like to do it from a crontab or other task scheduler on a different
host than my hadoop master. Does something like this exist? With or without
a pony :)

Thanks!

Re: Is there a network interface?

Posted by Dhruba Borthakur <dh...@gmail.com>.
> I attached a HiveLet (Made up term)

That's a cool name!

Re: Is there a network interface?

Posted by Gary Richardson <ga...@gmail.com>.
Hi Guys,

That's a big head start. It looks like I need to:

1) Configure Hive to use Derby as a meta db
2) Launch the hive thrift service with bin/hive --service hiveserver
3) Using the thrift api, I should be able to send queries from remote hosts

Am I missing anything from there?

Thanks!

On Thu, Feb 19, 2009 at 2:32 PM, Raghu Murthy <ra...@facebook.com> wrote:

> Hive supports both a Thrift service as well as a partial JDBC interface.
> Check out sample usage in service/src/test and jdbc/src/test. I can help
> you
> set up the thrift service if you have problems.
>
>
> On 2/19/09 2:16 PM, "Edward Capriolo" <ed...@gmail.com> wrote:
>
> > The best way to answer this is that all hadoop components work
> > remotely, assuming you have the proper configuration and library files
> > (the same ones from the remote cluster)
> >
> > I attached a HiveLet (Made up term). It was my first API testing
> > program. It is more or less a 'One Shot', run the query and exit
> > program.
> >
> > You need to run the Meta DB in Server mode for concurrent access.
> > http://wiki.apache.org/hadoop/HiveDerbyServerMode
> >
> > It is slightly complicated if your desktop is windows, but still doable.
> >
> > You would need:
> > Hadoop conf directory
> > Hive conf directory
> > hadoop distribution ( technically only jars )
> > Hive distribution ( technically only jars )
> >
> > When you start hadoop/hive they both pick up the locations of the
> > components from the configurations and start happily on a remote
> > machine. (Not counting firewall issues)
>
>

Re: Is there a network interface?

Posted by Raghu Murthy <ra...@facebook.com>.
Hive supports both a Thrift service as well as a partial JDBC interface.
Check out sample usage in service/src/test and jdbc/src/test. I can help you
set up the thrift service if you have problems.


On 2/19/09 2:16 PM, "Edward Capriolo" <ed...@gmail.com> wrote:

> The best way to answer this is that all hadoop components work
> remotely, assuming you have the proper configuration and library files
> (the same ones from the remote cluster)
> 
> I attached a HiveLet (Made up term). It was my first API testing
> program. It is more or less a 'One Shot', run the query and exit
> program.
> 
> You need to run the Meta DB in Server mode for concurrent access.
> http://wiki.apache.org/hadoop/HiveDerbyServerMode
> 
> It is slightly complicated if your desktop is windows, but still doable.
> 
> You would need:
> Hadoop conf directory
> Hive conf directory
> hadoop distribution ( technically only jars )
> Hive distribution ( technically only jars )
> 
> When you start hadoop/hive they both pick up the locations of the
> components from the configurations and start happily on a remote
> machine. (Not counting firewall issues)


Re: Is there a network interface?

Posted by Edward Capriolo <ed...@gmail.com>.
The best way to answer this is that all hadoop components work
remotely, assuming you have the proper configuration and library files
(the same ones from the remote cluster)

I attached a HiveLet (Made up term). It was my first API testing
program. It is more or less a 'One Shot', run the query and exit
program.

You need to run the Meta DB in Server mode for concurrent access.
http://wiki.apache.org/hadoop/HiveDerbyServerMode

It is slightly complicated if your desktop is windows, but still doable.

You would need:
Hadoop conf directory
Hive conf directory
hadoop distribution ( technically only jars )
Hive distribution ( technically only jars )

When you start hadoop/hive they both pick up the locations of the
components from the configurations and start happily on a remote
machine. (Not counting firewall issues)