Show job progress when using JDBC to run HIVE query

[Haijia Zhou <le...@gmail.com>]

Hi, All
 I have am writing a Hive client to run a Hive query using Hive JDBC driver.
 Since the data amount is huge I really would like to see the progress when
the query is running.
 Is there anyway I can get the job progress?
Thanks
Haijia

Re: Show job progress when using JDBC to run HIVE query

[Haijia Zhou <le...@gmail.com>]

Thanks a lot for all the answers and suggestions.
Looks like one hacky workaround is to check the hadoop task status. But for
my project it's way too much cost.


On Mon, Sep 17, 2012 at 8:31 AM, Bennie Schut <bs...@ebuddy.com> wrote:

> The jdbc driver uses thrift so if thrift can't then jdbc can't.
>
> This can be surprisingly difficult to do. Hive can split a query into x
> hadoop jobs and some will run in parallel and some will run in sequence.
> I've used oracle in the past (10 and 11) and I could also never find out
> how long a large job would take, which leads me to suspect it's not a
> trivial thing to do.
>
>
> -----Original Message-----
> From: MiaoMiao [mailto:liy099@gmail.com]
> Sent: Monday, September 17, 2012 6:17 AM
> To: user@hive.apache.org
> Subject: Re: Show job progress when using JDBC to run HIVE query
>
>
> Not familiar with JDBC, but thrift seems can't.
>
> On Sat, Sep 15, 2012 at 3:17 AM, Haijia Zhou <le...@gmail.com> wrote:
> > Hi, All
> >  I have am writing a Hive client to run a Hive query using Hive JDBC
> driver.
> >  Since the data amount is huge I really would like to see the progress
> > when the query is running.
> >  Is there anyway I can get the job progress?
> > Thanks
> > Haijia
>

RE: Show job progress when using JDBC to run HIVE query

[Bennie Schut <bs...@ebuddy.com>]

The jdbc driver uses thrift so if thrift can't then jdbc can't.

This can be surprisingly difficult to do. Hive can split a query into x hadoop jobs and some will run in parallel and some will run in sequence.
I've used oracle in the past (10 and 11) and I could also never find out how long a large job would take, which leads me to suspect it's not a trivial thing to do.


-----Original Message-----
From: MiaoMiao [mailto:liy099@gmail.com] 
Sent: Monday, September 17, 2012 6:17 AM
To: user@hive.apache.org
Subject: Re: Show job progress when using JDBC to run HIVE query


Not familiar with JDBC, but thrift seems can't.

On Sat, Sep 15, 2012 at 3:17 AM, Haijia Zhou <le...@gmail.com> wrote:
> Hi, All
>  I have am writing a Hive client to run a Hive query using Hive JDBC driver.
>  Since the data amount is huge I really would like to see the progress 
> when the query is running.
>  Is there anyway I can get the job progress?
> Thanks
> Haijia

Re: Show job progress when using JDBC to run HIVE query

[shashwat shriparv <dw...@gmail.com>]

One try you can give. As you run a hive query it in turn runs
a map-reduce at server, over there you can capture the progress of that map
and reduce percentage and send client side for progress bar or feedback

Regards

∞
Shashwat Shriparv




On Mon, Sep 17, 2012 at 9:47 AM, MiaoMiao <li...@gmail.com> wrote:

> Not familiar with JDBC, but thrift seems can't.
>
> On Sat, Sep 15, 2012 at 3:17 AM, Haijia Zhou <le...@gmail.com> wrote:
> > Hi, All
> >  I have am writing a Hive client to run a Hive query using Hive JDBC
> driver.
> >  Since the data amount is huge I really would like to see the progress
> when
> > the query is running.
> >  Is there anyway I can get the job progress?
> > Thanks
> > Haijia
>



-- 


∞
Shashwat Shriparv

Re: Show job progress when using JDBC to run HIVE query

[MiaoMiao <li...@gmail.com>]

Not familiar with JDBC, but thrift seems can't.

On Sat, Sep 15, 2012 at 3:17 AM, Haijia Zhou <le...@gmail.com> wrote:
> Hi, All
>  I have am writing a Hive client to run a Hive query using Hive JDBC driver.
>  Since the data amount is huge I really would like to see the progress when
> the query is running.
>  Is there anyway I can get the job progress?
> Thanks
> Haijia