You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Sultan Alamro <su...@gmail.com> on 2019/10/17 18:59:19 UTC
Run Reduce tasks without waiting for faild/killed Map tasks output
Hi,
I am running some experiments where not all Map tasks should be executed,
or continue processing. My application will fail/kill the running Map tasks
if a condition is true. The Reduce tasks should continue processing without
waiting for the output of the failed/killed tasks.
I was able to fail/kill the running Map tasks and let the AM not to recover
these tasks. However, the Reduce tasks keep waiting for the output of the
failed/killed tasks.
The question is, how can I let Reduce tasks continue processing with the
finished Map tasks only?
Thanks,