You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "ouyangwulin (Jira)" <ji...@apache.org> on 2022/12/19 09:45:00 UTC

[jira] [Commented] (FLINK-27925) Avoid to create watcher without the resourceVersion

    [ https://issues.apache.org/jira/browse/FLINK-27925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17649232#comment-17649232 ] 

ouyangwulin commented on FLINK-27925:
-------------------------------------

In the case of large-scale start and stop jobs, constantly reading data from etcd can cause a bottleneck in etcd performance. I agree with [~wangyang0918]  using informer will increase memory pressure. We can increase resourceversion=0 in watch to reduce data read from etcd.

> Avoid to create watcher without the resourceVersion
> ---------------------------------------------------
>
>                 Key: FLINK-27925
>                 URL: https://issues.apache.org/jira/browse/FLINK-27925
>             Project: Flink
>          Issue Type: Improvement
>          Components: Deployment / Kubernetes
>            Reporter: Aitozi
>            Priority: Major
>              Labels: pull-request-available
>
> Currently, we create the watcher in KubernetesResourceManager. But it do not pass the resourceVersion parameter, it will trigger a request to etcd. It will bring the burden to the etcd in large scale cluster (which have been seen in our internal k8s cluster). More detail can be found [here|https://kubernetes.io/docs/reference/using-api/api-concepts/#the-resourceversion-parameter] 
> I think we could use the informer to improve it (which will spawn a list-watch and maintain the resourceVersion internally)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)