You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by sh...@accenture.com on 2014/05/20 18:35:29 UTC
Any idea how to find out how long map reduce jobs are running?
Hi,
I need help in finding how long a map-reduce job runs. Any help is highly appreciated.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com
Re: Any idea how to find out how long map reduce jobs are running?
Posted by Brad Ruderman <br...@radiumone.com>.
Wrap the sqoop job in a python subprocess. Run that process on a separate
thread.
On Tue, May 20, 2014 at 9:57 AM, <sh...@accenture.com> wrote:
> There is no specific map reduce program here. Actually sqoop jobs run
> and internally that executes a map-reduce job.
>
>
>
> *From:* Brad Ruderman [mailto:bruderman@radiumone.com]
> *Sent:* Tuesday, May 20, 2014 9:54 AM
>
> *To:* user@hive.apache.org
> *Subject:* Re: Any idea how to find out how long map reduce jobs are
> running?
>
>
>
> Wrap your map reduce job in a script or something, maybe a python script
> and monitor the job asynchronously.
>
>
>
>
>
>
>
> On Tue, May 20, 2014 at 9:51 AM, <sh...@accenture.com> wrote:
>
> Thanks for the quick reply, Brad.
>
>
>
> Actually, the requirement is,
>
>
>
> We have to write a monitor to track how long a map-reduce job runs. If the
> map reduce job runs after a specific time, certain actions should be taken.
>
>
>
> Have you ever created one?
>
>
>
> Thanks,
>
> Shouvanik
>
>
>
> *From:* Brad Ruderman [mailto:bruderman@radiumone.com]
> *Sent:* Tuesday, May 20, 2014 9:38 AM
> *To:* user@hive.apache.org
> *Subject:* Re: Any idea how to find out how long map reduce jobs are
> running?
>
>
>
> Check jobtracker.
>
>
>
> Or you could do:
>
>
>
> mapred job -list
>
>
>
> Thanks,
>
> Brad
>
>
>
> On Tue, May 20, 2014 at 9:35 AM, <sh...@accenture.com> wrote:
>
> Hi,
>
>
>
> I need help in finding how long a map-reduce job runs. Any help is highly
> appreciated.
>
>
>
> Thanks,
>
> Shouvanik
>
>
> ------------------------------
>
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited. Where allowed
> by local law, electronic communications with Accenture and its affiliates,
> including e-mail and instant messaging (including content), may be scanned
> by our systems for the purposes of information security and assessment of
> internal compliance with Accenture policy.
>
> ______________________________________________________________________________________
>
> www.accenture.com
>
>
>
>
>
RE: Any idea how to find out how long map reduce jobs are running?
Posted by sh...@accenture.com.
There is no specific map reduce program here. Actually sqoop jobs run and internally that executes a map-reduce job.
From: Brad Ruderman [mailto:bruderman@radiumone.com]
Sent: Tuesday, May 20, 2014 9:54 AM
To: user@hive.apache.org
Subject: Re: Any idea how to find out how long map reduce jobs are running?
Wrap your map reduce job in a script or something, maybe a python script and monitor the job asynchronously.
On Tue, May 20, 2014 at 9:51 AM, <sh...@accenture.com>> wrote:
Thanks for the quick reply, Brad.
Actually, the requirement is,
We have to write a monitor to track how long a map-reduce job runs. If the map reduce job runs after a specific time, certain actions should be taken.
Have you ever created one?
Thanks,
Shouvanik
From: Brad Ruderman [mailto:bruderman@radiumone.com<ma...@radiumone.com>]
Sent: Tuesday, May 20, 2014 9:38 AM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: Any idea how to find out how long map reduce jobs are running?
Check jobtracker.
Or you could do:
mapred job -list
Thanks,
Brad
On Tue, May 20, 2014 at 9:35 AM, <sh...@accenture.com>> wrote:
Hi,
I need help in finding how long a map-reduce job runs. Any help is highly appreciated.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com>
Re: Any idea how to find out how long map reduce jobs are running?
Posted by Brad Ruderman <br...@radiumone.com>.
Wrap your map reduce job in a script or something, maybe a python script
and monitor the job asynchronously.
On Tue, May 20, 2014 at 9:51 AM, <sh...@accenture.com> wrote:
> Thanks for the quick reply, Brad.
>
>
>
> Actually, the requirement is,
>
>
>
> We have to write a monitor to track how long a map-reduce job runs. If the
> map reduce job runs after a specific time, certain actions should be taken.
>
>
>
> Have you ever created one?
>
>
>
> Thanks,
>
> Shouvanik
>
>
>
> *From:* Brad Ruderman [mailto:bruderman@radiumone.com]
> *Sent:* Tuesday, May 20, 2014 9:38 AM
> *To:* user@hive.apache.org
> *Subject:* Re: Any idea how to find out how long map reduce jobs are
> running?
>
>
>
> Check jobtracker.
>
>
>
> Or you could do:
>
>
>
> mapred job -list
>
>
>
> Thanks,
>
> Brad
>
>
>
> On Tue, May 20, 2014 at 9:35 AM, <sh...@accenture.com> wrote:
>
> Hi,
>
>
>
> I need help in finding how long a map-reduce job runs. Any help is highly
> appreciated.
>
>
>
> Thanks,
>
> Shouvanik
>
>
> ------------------------------
>
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited. Where allowed
> by local law, electronic communications with Accenture and its affiliates,
> including e-mail and instant messaging (including content), may be scanned
> by our systems for the purposes of information security and assessment of
> internal compliance with Accenture policy.
>
> ______________________________________________________________________________________
>
> www.accenture.com
>
>
>
RE: Any idea how to find out how long map reduce jobs are running?
Posted by sh...@accenture.com.
Thanks for the quick reply, Brad.
Actually, the requirement is,
We have to write a monitor to track how long a map-reduce job runs. If the map reduce job runs after a specific time, certain actions should be taken.
Have you ever created one?
Thanks,
Shouvanik
From: Brad Ruderman [mailto:bruderman@radiumone.com]
Sent: Tuesday, May 20, 2014 9:38 AM
To: user@hive.apache.org
Subject: Re: Any idea how to find out how long map reduce jobs are running?
Check jobtracker.
Or you could do:
mapred job -list
Thanks,
Brad
On Tue, May 20, 2014 at 9:35 AM, <sh...@accenture.com>> wrote:
Hi,
I need help in finding how long a map-reduce job runs. Any help is highly appreciated.
Thanks,
Shouvanik
________________________________
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________
www.accenture.com<http://www.accenture.com>
Re: Any idea how to find out how long map reduce jobs are running?
Posted by Brad Ruderman <br...@radiumone.com>.
Check jobtracker.
Or you could do:
mapred job -list
Thanks,
Brad
On Tue, May 20, 2014 at 9:35 AM, <sh...@accenture.com> wrote:
> Hi,
>
>
>
> I need help in finding how long a map-reduce job runs. Any help is highly
> appreciated.
>
>
>
> Thanks,
>
> Shouvanik
>
> ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited. Where allowed
> by local law, electronic communications with Accenture and its affiliates,
> including e-mail and instant messaging (including content), may be scanned
> by our systems for the purposes of information security and assessment of
> internal compliance with Accenture policy.
>
> ______________________________________________________________________________________
>
> www.accenture.com
>