You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Peter Sankauskas <pe...@motally.com> on 2010/02/02 18:42:42 UTC
How do I make an async call to Hive in Java?
Hi everyone,
I would like to execute a Hive query on the server in an asynchronous
manner. The Hive query will likely take a long time to complete, so I would
prefer not to block on the call. I am currently using Thirft to make a
blocking call (blocks on client.execute()), but I have not seen an example
of how to make a non-blocking call. Here is the blocking code:
TSocket transport = new TSocket("hive.example.com", 10000);
transport.setTimeout(999999999);
TBinaryProtocol protocol = new TBinaryProtocol(transport);
Client client = new ThriftHive.Client(protocol);
transport.open();
client.execute(hql); // Omitted HQL
List<String> rows;
while ((rows = client.fetchN(1000)) != null) {
for (String row : rows) {
// Do stuff with row
}
}
transport.close();
The code above is missing try/catch blocks to keep it short.
I can see that Thirft has things like "TNonblockingSocket" but I cannot find
any examples of how to use it, or if Hive even supports it. Does anyone have
any ideas how to do an async call? Is there a better way?
Thanks!
Kind regards,
Peter Sankauskas
Re: How do I make an async call to Hive in Java?
Posted by Zheng Shao <zs...@gmail.com>.
Hi Peter,
Currently there is no such method in Hive. It should be pretty
straightforward to do it with a thread.
I would encourage you to open a JIRA and post a patch if you are
interested in making it work.
Zheng
On Tue, Feb 2, 2010 at 9:42 AM, Peter Sankauskas <pe...@motally.com> wrote:
> Hi everyone,
>
> I would like to execute a Hive query on the server in an asynchronous
> manner. The Hive query will likely take a long time to complete, so I would
> prefer not to block on the call. I am currently using Thirft to make a
> blocking call (blocks on client.execute()), but I have not seen an example
> of how to make a non-blocking call. Here is the blocking code:
>
> TSocket transport = new TSocket("hive.example.com", 10000);
> transport.setTimeout(999999999);
> TBinaryProtocol protocol = new TBinaryProtocol(transport);
> Client client = new ThriftHive.Client(protocol);
> transport.open();
> client.execute(hql); // Omitted HQL
>
> List<String> rows;
> while ((rows = client.fetchN(1000)) != null) {
> for (String row : rows) {
> // Do stuff with row
> }
> }
>
> transport.close();
>
> The code above is missing try/catch blocks to keep it short.
>
> I can see that Thirft has things like "TNonblockingSocket" but I cannot find
> any examples of how to use it, or if Hive even supports it. Does anyone have
> any ideas how to do an async call? Is there a better way?
>
> Thanks!
>
> Kind regards,
> Peter Sankauskas
>
--
Yours,
Zheng