You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Terry Healy <th...@bnl.gov> on 2012/10/08 16:13:44 UTC
Collecting error messages from Mappers
Hi-
Is there a simple way for error output / debug information generated by
a Mapper to be collected in one place for a given M/R job run? I guess
what I'm hoping for is sort of the reverse of a Distributed Cache function.
Can anyone suggest an approach?
Thanks,
Terry
Re: Collecting error messages from Mappers
Posted by Bertrand Dechoux <de...@gmail.com>.
If it is only to get a parse overview of the run you can use counters. But
you shouldn't overuse them.
You can for example count exceptions by type (but by message is not a good
approach unless you are sure that the message is constant.)
Regards
Bertrand
On Mon, Oct 8, 2012 at 4:13 PM, Terry Healy <th...@bnl.gov> wrote:
> Hi-
>
> Is there a simple way for error output / debug information generated by
> a Mapper to be collected in one place for a given M/R job run? I guess
> what I'm hoping for is sort of the reverse of a Distributed Cache function.
>
> Can anyone suggest an approach?
>
> Thanks,
>
> Terry
>
--
Bertrand Dechoux
Re: Collecting error messages from Mappers
Posted by Bertrand Dechoux <de...@gmail.com>.
If it is only to get a parse overview of the run you can use counters. But
you shouldn't overuse them.
You can for example count exceptions by type (but by message is not a good
approach unless you are sure that the message is constant.)
Regards
Bertrand
On Mon, Oct 8, 2012 at 4:13 PM, Terry Healy <th...@bnl.gov> wrote:
> Hi-
>
> Is there a simple way for error output / debug information generated by
> a Mapper to be collected in one place for a given M/R job run? I guess
> what I'm hoping for is sort of the reverse of a Distributed Cache function.
>
> Can anyone suggest an approach?
>
> Thanks,
>
> Terry
>
--
Bertrand Dechoux
Re: Collecting error messages from Mappers
Posted by Bertrand Dechoux <de...@gmail.com>.
If it is only to get a parse overview of the run you can use counters. But
you shouldn't overuse them.
You can for example count exceptions by type (but by message is not a good
approach unless you are sure that the message is constant.)
Regards
Bertrand
On Mon, Oct 8, 2012 at 4:13 PM, Terry Healy <th...@bnl.gov> wrote:
> Hi-
>
> Is there a simple way for error output / debug information generated by
> a Mapper to be collected in one place for a given M/R job run? I guess
> what I'm hoping for is sort of the reverse of a Distributed Cache function.
>
> Can anyone suggest an approach?
>
> Thanks,
>
> Terry
>
--
Bertrand Dechoux
Re: Collecting error messages from Mappers
Posted by Bertrand Dechoux <de...@gmail.com>.
If it is only to get a parse overview of the run you can use counters. But
you shouldn't overuse them.
You can for example count exceptions by type (but by message is not a good
approach unless you are sure that the message is constant.)
Regards
Bertrand
On Mon, Oct 8, 2012 at 4:13 PM, Terry Healy <th...@bnl.gov> wrote:
> Hi-
>
> Is there a simple way for error output / debug information generated by
> a Mapper to be collected in one place for a given M/R job run? I guess
> what I'm hoping for is sort of the reverse of a Distributed Cache function.
>
> Can anyone suggest an approach?
>
> Thanks,
>
> Terry
>
--
Bertrand Dechoux