You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by VanHuy Pham <hu...@gmail.com> on 2012/07/05 20:35:37 UTC
Pulling off server's debug info to Client
Hi all,
I have been interested in getting the debug information and display it
to clients.
Specifically, when you submit a job to hive cli (on terminal say you
write "select * from somewhere ...) or through a thrift server, you will
see they hive displays some debug information on the screen (See below)
How I can I get this "live" information to the client while the
job is running so the client can see the stages of the job and how it
is going, see if the job gets killed, or fails... I am using thrift
server, and see this information on the thrift server's terminal and
wonder if I can transfer this info to thrift client. Or if it's
impossible to transfer this info to thrift client, how would one
direct this debug info somewhere to local file or mysql then the
information can be read from there?
Any ideas?
Thanks
------------------------------------------------------
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
---> getPaths = XXXXXXXXXXXX
Starting Job = XXXXXXXXXXXXXX, Tracking URL = XXXXXXXXXXXXXXXXXX
Kill Command = XXXXXXXXXXXXXXXXXXXX
Hadoop job information for Stage-1: number of mappers: 505; number of
reducers: 0
2012-07-04 01:52:42,145 Stage-1 map = 0%, reduce = 0%
2012-07-04 01:52:48,185 Stage-1 map = 5%, reduce = 0%, Cumulative CPU 42.92 sec
2012-07-04 01:52:49,196 Stage-1 map = 26%, reduce = 0%, Cumulative
CPU 213.03 sec
Re: Pulling off server's debug info to Client
Posted by Navis류승우 <na...@nexr.com>.
I've modified hive to notify jobId for monitoring purpose. Check the issue
and vote for acceptance ^^
https://issues.apache.org/jira/browse/HIVE-3235
2012/7/6 VanHuy Pham <hu...@gmail.com>
> Hi all,
> I have been interested in getting the debug information and display it
> to clients.
> Specifically, when you submit a job to hive cli (on terminal say you
> write "select * from somewhere ...) or through a thrift server, you will
> see they hive displays some debug information on the screen (See below)
>
> How I can I get this "live" information to the client while the
> job is running so the client can see the stages of the job and how it
> is going, see if the job gets killed, or fails... I am using thrift
> server, and see this information on the thrift server's terminal and
> wonder if I can transfer this info to thrift client. Or if it's
> impossible to transfer this info to thrift client, how would one
> direct this debug info somewhere to local file or mysql then the
> information can be read from there?
>
> Any ideas?
> Thanks
>
>
>
> ------------------------------------------------------
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> ---> getPaths = XXXXXXXXXXXX
> Starting Job = XXXXXXXXXXXXXX, Tracking URL = XXXXXXXXXXXXXXXXXX
> Kill Command = XXXXXXXXXXXXXXXXXXXX
> Hadoop job information for Stage-1: number of mappers: 505; number of
> reducers: 0
> 2012-07-04 01:52:42,145 Stage-1 map = 0%, reduce = 0%
> 2012-07-04 01:52:48,185 Stage-1 map = 5%, reduce = 0%, Cumulative CPU
> 42.92 sec
> 2012-07-04 01:52:49,196 Stage-1 map = 26%, reduce = 0%, Cumulative
> CPU 213.03 sec
>