problem with streaming map jobs not getting killed

[Andreas Kostyrka <an...@kostyrka.org>]

Hi!

I wonder if streaming.jar is anywhere logging the PID that it forks for the 
mapjob?

I'm having a problem with map jobs staying behind, and in the long run killing 
the node (via OOM), and now I naturally wonder under which conditions that 
can happen.

(yep, 5 map tasks that have been around for over 24 hours, while my nodes have 
a maximum of 2 map tasks configured)

Andreas

RE: problem with streaming map jobs not getting killed

[Devaraj Das <dd...@yahoo-inc.com>]

No the PID is not logged. So is it the framework side java tasks not getting
killed or is it the Streaming children? By the way, the handling of process
groups should be handled better when we have HADOOP-1380. 

> -----Original Message-----
> From: Andreas Kostyrka [mailto:andreas@kostyrka.org] 
> Sent: Monday, June 09, 2008 7:56 PM
> To: core-user@hadoop.apache.org
> Subject: problem with streaming map jobs not getting killed
> 
> Hi!
> 
> I wonder if streaming.jar is anywhere logging the PID that it 
> forks for the mapjob?
> 
> I'm having a problem with map jobs staying behind, and in the 
> long run killing the node (via OOM), and now I naturally 
> wonder under which conditions that can happen.
> 
> (yep, 5 map tasks that have been around for over 24 hours, 
> while my nodes have a maximum of 2 map tasks configured)
> 
> Andreas
>