hanging map reduce processes

[Henning Blohm <he...@zfabrik.de>]

Hi,

found that several people have run into this issue, but I was not able 
to find a solution yet.

We have reduce tasks that leave a hanging "child" process. The 
implementation uses a lot of third party stuff and leave Timer threads 
running (as you can readily see in thread dumps). Which is bad style - 
no doubt. But eventually we don't really care - when the reduce is done, 
its done and the process should be really just killed rather than 
hanging around and eventually impacting the cluster.

Is there a way to force killing of child processes, e.g. based on job 
configuration?

Thanks,
   Henning

Re: hanging map reduce processes

[Harsh J <ha...@cloudera.com>]

Hey Henning,

What version of Hadoop are you running, and can we have a dumbed down
sample to reproduce?

On Thu, Oct 27, 2011 at 3:28 PM, Henning Blohm <he...@zfabrik.de> wrote:
> Hi,
>
> found that several people have run into this issue, but I was not able to
> find a solution yet.
>
> We have reduce tasks that leave a hanging "child" process. The
> implementation uses a lot of third party stuff and leave Timer threads
> running (as you can readily see in thread dumps). Which is bad style - no
> doubt. But eventually we don't really care - when the reduce is done, its
> done and the process should be really just killed rather than hanging around
> and eventually impacting the cluster.
>
> Is there a way to force killing of child processes, e.g. based on job
> configuration?
>
> Thanks,
>  Henning
>



-- 
Harsh J