You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Salva Alcántara <sa...@gmail.com> on 2025/03/04 08:29:20 UTC
Flink K8 Operator: New Snapshot CRD Not Working
Hey all! I recently bumped the Flink Kubernetes Operator to v1.10.0 and one
of the things I wanted to check is the usage of the new FlinkStateSnapshot
CRD. I confirmed that the CRD was correctly created in my cluster, however
I'm still seeing these logs:
```
Starting Operator
2025-03-01T08:31:08.779422Z main ERROR appender CONSOLE has no parameter
that matches element JsonLayout
2025-03-01T08:31:08.782927Z main ERROR Unable to locate appender
"ConsoleAppender" for logger config "root"
2025-03-01 08:31:12,885 i.f.k.c.d.i.VersionUsageUtils [WARN ] The client
is using resource type 'flinkstatesnapshots' with unstable version 'v1beta1'
2025-03-01 08:31:14,180 o.a.f.k.o.c.FlinkConfigManager [WARN ]
FlinkStateSnapshot CRD was not installed, snapshot resources will be
disabled!
```
I think this relates to the RBAC stuff. For what it's worth, the
FlinkStateSnapshot CRD was not installed log message goes away if I switch
to a cluster-wide installaction (which handles RBAC via clusterrole &
clusterrolebinding). However, for a namespaced installation like mine
(using a non-empty array for watchNamespaces) there must be something
wrong, despite RBAC apparently being right, i.e.:
```
kubectl auth can-i list flinkstatesnapshot -n a-watched-namespace
--as=system:serviceaccount:flink-operator:flink-operator
yes
```
The answer is the same for any namespace within watchNamespaces (w.r.t.
flink-operator, which is where I deploy the operator).
The issue might be in this line:
-
https://github.com/apache/flink-kubernetes-operator/blob/9eb3c385b90a5a2f08376720f[…]ache/flink/kubernetes/operator/utils/KubernetesClientUtils.java
<https://github.com/apache/flink-kubernetes-operator/blob/9eb3c385b90a5a2f08376720f3204d1784981a0c/flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/KubernetesClientUtils.java#L72C31-L72C67>
which is not passing any special config, maybe the idea was to use
getKubernetesClient instead? Can anyone help troubleshoot the issue?
Re: Flink K8 Operator: New Snapshot CRD Not Working
Posted by Salva Alcántara <sa...@gmail.com>.
Thanks a lot for the confirmation Mate!
Makes sense, in the meantime I fixed the issue by adding a ClusterRole &
ClusterRoleBinding for the flink-operator ServiceAccount.
Regards,
Salva
On Sun, Mar 9, 2025 at 2:11 PM Mate Czagany <cz...@gmail.com> wrote:
> Hi!
>
> When not using a cluster-wide installation, Role and RoleBinding resources
> to be able to interact with Flink resources are only installed in the
> namespaces listed in the `watchNamespaces` field. If the namespace where
> the operator is installed is not added to this list, RBAC will be
> configured in a way that doesn't allow for Flink resources (one of them
> being FlinkStateSnapshot) to be listed in that namespace. I have overlooked
> this when implementing this check.
>
> In the next release we should remove this check as it is as I think we can
> expect that all installations will have the FlinkStateSnapshot CRD. For
> now, you can fix it by creating a new Role and RoleBinding for the
> flink-operator ServiceAccount or by editing the flink-operator Role in the
> namespace where the operator Helm chart was installed, adding this to its
> rules:
>
> - apiGroups:
> - flink.apache.org
> resources:
> - flinkstatesnapshots
> verbs:
> - list
>
> In simple terms, the method you have linked needs to be able to list the
> FlinkStateSnapshot resources in the namespace the operator is running in.
> And by using `watchNamespaces` and installing the operator in a namespace
> not listed there, the operator won't have access to list these resources.
>
> Salva Alcántara <sa...@gmail.com> ezt írta (időpont: 2025. márc.
> 4., K, 9:30):
>
>> Hey all! I recently bumped the Flink Kubernetes Operator to v1.10.0 and
>> one of the things I wanted to check is the usage of the new
>> FlinkStateSnapshot CRD. I confirmed that the CRD was correctly created in
>> my cluster, however I'm still seeing these logs:
>>
>> ```
>> Starting Operator
>> 2025-03-01T08:31:08.779422Z main ERROR appender CONSOLE has no parameter
>> that matches element JsonLayout
>> 2025-03-01T08:31:08.782927Z main ERROR Unable to locate appender
>> "ConsoleAppender" for logger config "root"
>> 2025-03-01 08:31:12,885 i.f.k.c.d.i.VersionUsageUtils [WARN ] The client
>> is using resource type 'flinkstatesnapshots' with unstable version 'v1beta1'
>> 2025-03-01 08:31:14,180 o.a.f.k.o.c.FlinkConfigManager [WARN ]
>> FlinkStateSnapshot CRD was not installed, snapshot resources will be
>> disabled!
>> ```
>>
>> I think this relates to the RBAC stuff. For what it's worth, the
>> FlinkStateSnapshot CRD was not installed log message goes away if I switch
>> to a cluster-wide installaction (which handles RBAC via clusterrole &
>> clusterrolebinding). However, for a namespaced installation like mine
>> (using a non-empty array for watchNamespaces) there must be something
>> wrong, despite RBAC apparently being right, i.e.:
>>
>> ```
>> kubectl auth can-i list flinkstatesnapshot -n a-watched-namespace
>> --as=system:serviceaccount:flink-operator:flink-operator
>> yes
>> ```
>>
>> The answer is the same for any namespace within watchNamespaces (w.r.t.
>> flink-operator, which is where I deploy the operator).
>>
>> The issue might be in this line:
>>
>> -
>> https://github.com/apache/flink-kubernetes-operator/blob/9eb3c385b90a5a2f08376720f[…]ache/flink/kubernetes/operator/utils/KubernetesClientUtils.java
>> <https://github.com/apache/flink-kubernetes-operator/blob/9eb3c385b90a5a2f08376720f3204d1784981a0c/flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/KubernetesClientUtils.java#L72C31-L72C67>
>>
>> which is not passing any special config, maybe the idea was to use
>> getKubernetesClient instead? Can anyone help troubleshoot the issue?
>>
>
Re: Flink K8 Operator: New Snapshot CRD Not Working
Posted by Mate Czagany <cz...@gmail.com>.
Hi!
When not using a cluster-wide installation, Role and RoleBinding resources
to be able to interact with Flink resources are only installed in the
namespaces listed in the `watchNamespaces` field. If the namespace where
the operator is installed is not added to this list, RBAC will be
configured in a way that doesn't allow for Flink resources (one of them
being FlinkStateSnapshot) to be listed in that namespace. I have overlooked
this when implementing this check.
In the next release we should remove this check as it is as I think we can
expect that all installations will have the FlinkStateSnapshot CRD. For
now, you can fix it by creating a new Role and RoleBinding for the
flink-operator ServiceAccount or by editing the flink-operator Role in the
namespace where the operator Helm chart was installed, adding this to its
rules:
- apiGroups:
- flink.apache.org
resources:
- flinkstatesnapshots
verbs:
- list
In simple terms, the method you have linked needs to be able to list the
FlinkStateSnapshot resources in the namespace the operator is running in.
And by using `watchNamespaces` and installing the operator in a namespace
not listed there, the operator won't have access to list these resources.
Salva Alcántara <sa...@gmail.com> ezt írta (időpont: 2025. márc.
4., K, 9:30):
> Hey all! I recently bumped the Flink Kubernetes Operator to v1.10.0 and
> one of the things I wanted to check is the usage of the new
> FlinkStateSnapshot CRD. I confirmed that the CRD was correctly created in
> my cluster, however I'm still seeing these logs:
>
> ```
> Starting Operator
> 2025-03-01T08:31:08.779422Z main ERROR appender CONSOLE has no parameter
> that matches element JsonLayout
> 2025-03-01T08:31:08.782927Z main ERROR Unable to locate appender
> "ConsoleAppender" for logger config "root"
> 2025-03-01 08:31:12,885 i.f.k.c.d.i.VersionUsageUtils [WARN ] The client
> is using resource type 'flinkstatesnapshots' with unstable version 'v1beta1'
> 2025-03-01 08:31:14,180 o.a.f.k.o.c.FlinkConfigManager [WARN ]
> FlinkStateSnapshot CRD was not installed, snapshot resources will be
> disabled!
> ```
>
> I think this relates to the RBAC stuff. For what it's worth, the
> FlinkStateSnapshot CRD was not installed log message goes away if I switch
> to a cluster-wide installaction (which handles RBAC via clusterrole &
> clusterrolebinding). However, for a namespaced installation like mine
> (using a non-empty array for watchNamespaces) there must be something
> wrong, despite RBAC apparently being right, i.e.:
>
> ```
> kubectl auth can-i list flinkstatesnapshot -n a-watched-namespace
> --as=system:serviceaccount:flink-operator:flink-operator
> yes
> ```
>
> The answer is the same for any namespace within watchNamespaces (w.r.t.
> flink-operator, which is where I deploy the operator).
>
> The issue might be in this line:
>
> -
> https://github.com/apache/flink-kubernetes-operator/blob/9eb3c385b90a5a2f08376720f[…]ache/flink/kubernetes/operator/utils/KubernetesClientUtils.java
> <https://github.com/apache/flink-kubernetes-operator/blob/9eb3c385b90a5a2f08376720f3204d1784981a0c/flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/KubernetesClientUtils.java#L72C31-L72C67>
>
> which is not passing any special config, maybe the idea was to use
> getKubernetesClient instead? Can anyone help troubleshoot the issue?
>