You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Stephan Ewen (Jira)" <ji...@apache.org> on 2021/09/21 10:16:00 UTC
[jira] [Closed] (FLINK-24303) SourceCoordinator exception may fail
Session Cluster
[ https://issues.apache.org/jira/browse/FLINK-24303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephan Ewen closed FLINK-24303.
--------------------------------
> SourceCoordinator exception may fail Session Cluster
> ----------------------------------------------------
>
> Key: FLINK-24303
> URL: https://issues.apache.org/jira/browse/FLINK-24303
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Common
> Reporter: Seth Wiesman
> Assignee: Stephan Ewen
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.13.3, 1.15.0, 1.14.1
>
>
> The SourceCoordinator currently forwards all exceptions from `Source#createEnumerator` up the stack triggering a JobMaster failover. However, JobMaster failover only works if HA is enabled[1]. If HA is not enabled the fatal error handler will simply exit the JM process killing the entire cluster. This is problematic in the case of a session cluster where there may be multiple jobs running. It also does not play well with external tooling that does not expect job failure to cause a full cluster failure.
>
> It would be preferable if failure to create an enumerator did not take down the entire cluster, but instead failed that particular job.
>
> [1] [https://github.com/apache/flink/blob/7f69331294ab2ab73f77b40a4320cdda53246afe/flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java#L898-L903]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)