You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "r. r." <ro...@abv.bg> on 2017/09/29 15:10:47 UTC
state of parallel jobs when one task fails
Hello
I have a simple job with a single map() processing which I want to run with many documents in parallel in Flink.
What will happen if one of the 'instances' of the job fails?
This statement in Flink docs confuses me:
"In case of failures, a job switches first to failing where it cancels all running tasks".
So if I have 10 documents processed in parallel in the job's map() (each in a different task slot, I presume) and one of them fails, does it mean that all the rest will be failed/cancelled as well?
Thanks!
Re: state of parallel jobs when one task fails
Posted by "r. r." <ro...@abv.bg>.
Thanks a lot - wasn't aware of FailoverStrategy
Best regards
Robert
>-------- Оригинално писмо --------
>От: Piotr Nowojski piotr@data-artisans.com
>Относно: Re: state of parallel jobs when one task fails
>До: "r. r." <ro...@abv.bg>
>Изпратено на: 29.09.2017 18:21
>
>
>
>
> Hi,
>
>
>
>
>
> Yes, by default Flink will restart all of the tasks. I think that since Flink 1.3, you can configure a
> FailoverStrategy to change this behavior.
>
>
>
>
>
> Thanks, Piotrek
>
>
>
>
>
> On Sep 29, 2017, at 5:10 PM, r. r. <
> robert@abv.bg> wrote:
>
>
>
>
> Hello
> I have a simple job with a single map() processing which I want to run with many documents in parallel in Flink.
> What will happen if one of the 'instances' of the job fails?
>
> This statement in Flink docs confuses me:
> "In case of failures, a job switches first to failing where it cancels all running tasks".
> So if I have 10 documents processed in parallel in the job's map() (each in a different task slot, I presume) and one of them fails, does it mean that all the rest will be failed/cancelled as well?
>
> Thanks!
>
>
>
>
>
>
>
>
Re: state of parallel jobs when one task fails
Posted by Piotr Nowojski <pi...@data-artisans.com>.
Hi,
Yes, by default Flink will restart all of the tasks. I think that since Flink 1.3, you can configure a FailoverStrategy <https://ci.apache.org/projects/flink/flink-docs-release-1.3/api/java/org/apache/flink/runtime/executiongraph/failover/FailoverStrategy.html> to change this behavior.
Thanks, Piotrek
> On Sep 29, 2017, at 5:10 PM, r. r. <ro...@abv.bg> wrote:
>
> Hello
> I have a simple job with a single map() processing which I want to run with many documents in parallel in Flink.
> What will happen if one of the 'instances' of the job fails?
>
> This statement in Flink docs confuses me:
> "In case of failures, a job switches first to failing where it cancels all running tasks".
> So if I have 10 documents processed in parallel in the job's map() (each in a different task slot, I presume) and one of them fails, does it mean that all the rest will be failed/cancelled as well?
>
> Thanks!
>