How to pause a job?

[xiao yang <ya...@gmail.com>]

Hi, all

I'm running a job on a 10-nodes cluster.
Now I want to add another node.
I have to reconfigure the dfs, and restart it, but I don't want to
stop the running job. It run for 1 week already.
What should I do? Is there a way to pause a job, and resume it after
dfs restart.

Thanks!
Xiao

Re: How to pause a job?

[xiao yang <ya...@gmail.com>]

Thanks for reply. The job finally failed. It seems that I should make
sure the job won't take too long next time.

On 12/7/09, Rekha Joshi <re...@yahoo-inc.com> wrote:
>
> If you have workflow engine around your job, that can take care of
> restarting the jobs. For direct hadoop execution,  please refer
> MAPREDUCE-227/ MAPREDUCE-828
>
> Not sure if it would work, you might try (effectively) blocking this job run
> by -set-priority on this job to very low and have other VHP jobs running..
> Thanks!
>
> On 12/6/09 5:02 PM, "xiao yang" <ya...@gmail.com> wrote:
>
> Hi, all
>
> I'm running a job on a 10-nodes cluster.
> Now I want to add another node.
> I have to reconfigure the dfs, and restart it, but I don't want to
> stop the running job. It run for 1 week already.
> What should I do? Is there a way to pause a job, and resume it after
> dfs restart.
>
> Thanks!
> Xiao
>
>

Re: How to pause a job?

[Rekha Joshi <re...@yahoo-inc.com>]

If you have workflow engine around your job, that can take care of restarting the jobs. For direct hadoop execution,  please refer MAPREDUCE-227/ MAPREDUCE-828

Not sure if it would work, you might try (effectively) blocking this job run by -set-priority on this job to very low and have other VHP jobs running.. Thanks!

On 12/6/09 5:02 PM, "xiao yang" <ya...@gmail.com> wrote:

Hi, all

I'm running a job on a 10-nodes cluster.
Now I want to add another node.
I have to reconfigure the dfs, and restart it, but I don't want to
stop the running job. It run for 1 week already.
What should I do? Is there a way to pause a job, and resume it after
dfs restart.

Thanks!
Xiao