You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Sultan Alamro <su...@gmail.com> on 2019/10/17 18:59:19 UTC

Run Reduce tasks without waiting for faild/killed Map tasks output

Hi,

I am running some experiments where not all Map tasks should be executed,
or continue processing. My application will fail/kill the running Map tasks
if a condition is true. The Reduce tasks should continue processing without
waiting for the output of the failed/killed tasks.

I was able to fail/kill the running Map tasks and let the AM not to recover
these tasks. However, the Reduce tasks keep waiting for the output of the
failed/killed tasks.

The question is, how can I let Reduce tasks continue processing with the
finished Map tasks only?

Thanks,