You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Neelishaa Srivastava (Jira)" <ji...@apache.org> on 2022/07/04 09:31:00 UTC
[jira] [Comment Edited] (FLINK-25098) Jobmanager CrashLoopBackOff in HA configuration
[ https://issues.apache.org/jira/browse/FLINK-25098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17562099#comment-17562099 ]
Neelishaa Srivastava edited comment on FLINK-25098 at 7/4/22 9:30 AM:
----------------------------------------------------------------------
Hi Till Rohrmann ,
can you please guide us to the similar issue jobmanager pod is stuck in CrashLoopBackOff state seen .The logs are already mentioned by MAU CHEE YEN in the above comment .
was (Author: JIRAUSER292214):
Hi Till Rohrmann ,
can you please guide us to the similar issue jobmanager pod is stuck in CrashLoopBackOff state seen .The logs are already mentioned by MAU CHEE YEN on the above comment .
> Jobmanager CrashLoopBackOff in HA configuration
> -----------------------------------------------
>
> Key: FLINK-25098
> URL: https://issues.apache.org/jira/browse/FLINK-25098
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Kubernetes
> Affects Versions: 1.13.2, 1.13.3
> Environment: Reproduced with:
> * Persistent jobs storage provided by the rocks-cephfs storage class.
> * OpenShift 4.9.5.
> Reporter: Adrian Vasiliu
> Priority: Critical
> Attachments: JM-FlinkException-checkpointHA.txt, flink_checkpoint_issue.txt, iaf-insights-engine--7fc4-eve-29ee-ep-jobmanager-1-jobmanager.log, jm-flink-ha-jobmanager-log.txt, jm-flink-ha-tls-proxy-log.txt
>
>
> In a Kubernetes deployment of Flink 1.13.2 (also reproduced with Flink 1.13.3), turning to Flink HA by using 3 replicas of the jobmanager leads to CrashLoopBackoff for all replicas.
> Attaching the full logs of the {{jobmanager}} and {{tls-proxy}} containers of jobmanager pod:
> [^jm-flink-ha-jobmanager-log.txt]
> [^jm-flink-ha-tls-proxy-log.txt]
> Reproduced with:
> * Persistent jobs storage provided by the {{rocks-cephfs}} storage class (shared by all replicas - ReadWriteMany) and mount path set via {{{}high-availability.storageDir: file///<dir>{}}}.
> * OpenShift 4.9.5 and also 4.8.x - reproduced in several clusters, it's not a "one-shot" trouble.
> Remarks:
> * This is a follow-up of https://issues.apache.org/jira/browse/FLINK-22014?focusedCommentId=17450524&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17450524.
> * Picked Critical severity as HA is critical for our product.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)