You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yi Tang (Jira)" <ji...@apache.org> on 2021/03/29 10:10:00 UTC
[jira] [Commented] (FLINK-22006) Could not run more than 20 jobs in
a native K8s session when K8s HA enabled
[ https://issues.apache.org/jira/browse/FLINK-22006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17310546#comment-17310546 ]
Yi Tang commented on FLINK-22006:
---------------------------------
[~fly_in_gis] Did you reproduce the issue? I think it's easy to reproduce.
And the problem is definitely about the okhttp connectionPool. Fabric8 kubernetes client will create a separate webSocket connection for each config map watching. So after it exhausting all connections in the connection pool, new webSocket request can not be established. Then the new Job can not be submitted successfully.
So my advice is creating a single config map watching and dispatching events for different usages if possible and if we still using the fabric8 kubernetes client.
> Could not run more than 20 jobs in a native K8s session when K8s HA enabled
> ---------------------------------------------------------------------------
>
> Key: FLINK-22006
> URL: https://issues.apache.org/jira/browse/FLINK-22006
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 1.12.2, 1.13.0
> Reporter: Yang Wang
> Priority: Critical
> Labels: k8s-ha
> Attachments: image-2021-03-24-18-08-42-116.png
>
>
> Currently, if we start a native K8s session cluster when K8s HA enabled, we could not run more than 20 streaming jobs.
>
> The latest job is always initializing, and the previous one is created and waiting to be assigned. It seems that some internal resources have been exhausted, e.g. okhttp thread pool , tcp connections or something else.
> !image-2021-03-24-18-08-42-116.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)