You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Ondřej Klimpera <kl...@fit.cvut.cz> on 2012/04/18 14:39:07 UTC
Setting a timeout for one Map() input processing
Hello, I'd like to ask you if there is a possibility of setting a
timeout for processing one input line of text input in mapper function.
The idea is, that if processing of one line is too long, Hadoop will cut
this process and continue processing next input line.
Thank you for your answer.
Ondrej Klimpera
Re: Setting a timeout for one Map() input processing
Posted by Michel Segel <mi...@hotmail.com>.
Multiple threads within the mapper where you have the main thread starting a timeout thread and a process thread. Take the result of the thread that finishes first, ignoring the other and killing it all within the Mapper.map() method? Sure it seems possible.
(you output from the time out thread is a NOOP...)
Sent from a remote device. Please excuse any typos...
Mike Segel
On Apr 18, 2012, at 7:39 AM, Ondřej Klimpera <kl...@fit.cvut.cz> wrote:
> Hello, I'd like to ask you if there is a possibility of setting a timeout for processing one input line of text input in mapper function.
>
> The idea is, that if processing of one line is too long, Hadoop will cut this process and continue processing next input line.
>
> Thank you for your answer.
>
> Ondrej Klimpera
>
Re: Setting a timeout for one Map() input processing
Posted by Ondřej Klimpera <kl...@fit.cvut.cz>.
Thanks, I'll try to implement it and get you know if it worked.
On 04/18/2012 04:07 PM, Harsh J wrote:
> Since you're looking for per-line (and not per-task/file) monitoring,
> this is best done by your own application code (a monitoring thread,
> etc.).
>
> On Wed, Apr 18, 2012 at 6:09 PM, Ondřej Klimpera<kl...@fit.cvut.cz> wrote:
>> Hello, I'd like to ask you if there is a possibility of setting a timeout
>> for processing one input line of text input in mapper function.
>>
>> The idea is, that if processing of one line is too long, Hadoop will cut
>> this process and continue processing next input line.
>>
>> Thank you for your answer.
>>
>> Ondrej Klimpera
>
>
Re: Setting a timeout for one Map() input processing
Posted by Harsh J <ha...@cloudera.com>.
Since you're looking for per-line (and not per-task/file) monitoring,
this is best done by your own application code (a monitoring thread,
etc.).
On Wed, Apr 18, 2012 at 6:09 PM, Ondřej Klimpera <kl...@fit.cvut.cz> wrote:
> Hello, I'd like to ask you if there is a possibility of setting a timeout
> for processing one input line of text input in mapper function.
>
> The idea is, that if processing of one line is too long, Hadoop will cut
> this process and continue processing next input line.
>
> Thank you for your answer.
>
> Ondrej Klimpera
--
Harsh J