You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Nikhil Goyal <no...@gmail.com> on 2018/02/20 22:52:30 UTC

Job never finishing

Hi guys,

I have a job which gets stuck if a couple of tasks get killed due to OOM
exception. Spark doesn't kill the job and it keeps on running for hours.
Ideally I would expect Spark to kill the job or restart the killed
executors but nothing seems to be happening. Anybody got idea about this?

Thanks
Nikhil

Re: Job never finishing

Posted by Nikhil Goyal <no...@gmail.com>.

Thanks for the help :)

On Tue, Feb 20, 2018 at 4:22 PM, Femi Anthony <fe...@gmail.com> wrote:

> You can use spark speculation as a way to get around the problem.
>
> Here is a useful link:
>
> http://asyncified.io/2016/08/13/leveraging-spark-
> speculation-to-identify-and-re-schedule-slow-running-tasks/
>
> Sent from my iPhone
>
> On Feb 20, 2018, at 5:52 PM, Nikhil Goyal <no...@gmail.com> wrote:
>
> Hi guys,
>
> I have a job which gets stuck if a couple of tasks get killed due to OOM
> exception. Spark doesn't kill the job and it keeps on running for hours.
> Ideally I would expect Spark to kill the job or restart the killed
> executors but nothing seems to be happening. Anybody got idea about this?
>
> Thanks
> Nikhil
>
>

Re: Job never finishing

Posted by Femi Anthony <fe...@gmail.com>.

You can use spark speculation as a way to get around the problem.

Here is a useful link:

http://asyncified.io/2016/08/13/leveraging-spark-speculation-to-identify-and-re-schedule-slow-running-tasks/

Sent from my iPhone

> On Feb 20, 2018, at 5:52 PM, Nikhil Goyal <no...@gmail.com> wrote:
> 
> Hi guys,
> 
> I have a job which gets stuck if a couple of tasks get killed due to OOM exception. Spark doesn't kill the job and it keeps on running for hours. Ideally I would expect Spark to kill the job or restart the killed executors but nothing seems to be happening. Anybody got idea about this?
> 
> Thanks
> Nikhil