Override automatica final streaming job status message?

[Keith Wiley <kw...@keithwiley.com>]

Streaming tasks seem to get a final status message from Hadoop itself (not my code) that reports "R/W", presumably the number of records read and written (the number of lines of text I assume).  Problem is, I like to display a final status message for my tasks.  That way I can quickly glance at a list of hundreds of mappers on the job tracker and see a broad survey of their final statuses.  This works fine in an all-
Java job, but in streaming, this R/W status is automatically generated, thus overwriting my final status.

Is there anything I can do about this?

Thank you.

________________________________________________________________________________
Keith Wiley               kwiley@keithwiley.com               www.keithwiley.com

"And what if we picked the wrong religion?  Every week, we're just making God
madder and madder!"
  -- Homer Simpson
________________________________________________________________________________

Re: Override automatica final streaming job status message?

[Mahadev Konar <ma...@apache.org>]

Keith,
  I cant see how you can do it. The status is set in PipeMapred.java
which is the streaming plugin that talks to mapreduce framework and
generated this status message. I cant think of neway of hacking arnd
it except for having your own streaming jar :).

thanks
mahadev

On Fri, Feb 4, 2011 at 2:49 PM, Keith Wiley <kw...@keithwiley.com> wrote:
> Streaming tasks seem to get a final status message from Hadoop itself (not my code) that reports "R/W", presumably the number of records read and written (the number of lines of text I assume).  Problem is, I like to display a final status message for my tasks.  That way I can quickly glance at a list of hundreds of mappers on the job tracker and see a broad survey of their final statuses.  This works fine in an all-
> Java job, but in streaming, this R/W status is automatically generated, thus overwriting my final status.
>
> Is there anything I can do about this?
>
> Thank you.
>
> ________________________________________________________________________________
> Keith Wiley               kwiley@keithwiley.com               www.keithwiley.com
>
> "And what if we picked the wrong religion?  Every week, we're just making God
> madder and madder!"
>  -- Homer Simpson
> ________________________________________________________________________________
>
>
>
>