[jira] [Comment Edited] (FLINK-31135) ConfigMap DataSize went > 1 MB and cluster stopped working

["Zhihao Chen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-31135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712854#comment-17712854 ] 

Zhihao Chen edited comment on FLINK-31135 at 4/17/23 1:48 AM:
--------------------------------------------------------------

I have encountered the same issue. Actually, it's an ongoing issue for us. I believe it has nothing to do with the Flink-Kubernetes-operator as it happened with both Flink Standalone Kubernetes deployment and Flink-kubernetes-operator deployment.

 

I have checked our configuration but didn't find anything interesting.


was (Author: JIRAUSER299871):
I have encountered the same issue. Actually, it's an ongoing issue for us. I believe it has nothing to do with the Flink-Kubernetes-operator as it happened with Flink Standalone Kubernetes deployment and Flink-kubernetes-operator deployment.

 

I have checked our configuration but didn't find anything interesting.

> ConfigMap DataSize went > 1 MB and cluster stopped working
> ----------------------------------------------------------
>
>                 Key: FLINK-31135
>                 URL: https://issues.apache.org/jira/browse/FLINK-31135
>             Project: Flink
>          Issue Type: Bug
>          Components: Kubernetes Operator
>    Affects Versions: kubernetes-operator-1.2.0
>            Reporter: Sriram Ganesh
>            Priority: Major
>
> I am Flink Operator to manage clusters. Flink version: 1.15.2. Flink jobs failed with the below error. It seems the config map size went beyond 1 MB (default size). 
> Since it is managed by the operator and config maps are not updated with any manual intervention, I suspect it could be an operator issue. 
>  
> {code:java}
> Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: PUT at: https://<IP>/api/v1/namespaces/<NS>/configmaps/<job>-config-map. Message: ConfigMap "<job>-config-map" is invalid: []: Too long: must have at most 1048576 bytes. Received status: Status(apiVersion=v1, code=422, details=StatusDetails(causes=[StatusCause(field=[], message=Too long: must have at most 1048576 bytes, reason=FieldValueTooLong, additionalProperties={})], group=null, kind=ConfigMap, name=<job>-config-map, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=ConfigMap "<job>-config-map" is invalid: []: Too long: must have at most 1048576 bytes, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Invalid, status=Failure, additionalProperties={}).
> at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:673) ~[flink-dist-1.15.2.jar:1.15.2]
> at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:612) ~[flink-dist-1.15.2.jar:1.15.2]
> at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:560) ~[flink-dist-1.15.2.jar:1.15.2]
> at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:521) ~[flink-dist-1.15.2.jar:1.15.2]
> at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:347) ~[flink-dist-1.15.2.jar:1.15.2]
> at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:327) ~[flink-dist-1.15.2.jar:1.15.2]
> at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleUpdate(BaseOperation.java:781) ~[flink-dist-1.15.2.jar:1.15.2]
> at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.lambda$replace$1(HasMetadataOperation.java:183) ~[flink-dist-1.15.2.jar:1.15.2]
> at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:188) ~[flink-dist-1.15.2.jar:1.15.2]
> at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:130) ~[flink-dist-1.15.2.jar:1.15.2]
> at io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:41) ~[flink-dist-1.15.2.jar:1.15.2]
> at org.apache.flink.kubernetes.kubeclient.Fabric8FlinkKubeClient.lambda$attemptCheckAndUpdateConfigMap$11(Fabric8FlinkKubeClient.java:325) ~[flink-dist-1.15.2.jar:1.15.2]
> at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700) ~[?:?]
> ... 3 more {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)