You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Vignesh Kumar Kathiresan via user <us...@flink.apache.org> on 2022/10/27 21:20:31 UTC

Flink Kubernetes rocksdb backend volume type

Hi all,

What is the recommended practice of volume mounting for the rocksdb backend
in a flink kubernetes standalone setup.

We currently have a "local"  type. We have local persistent volumes and
persistent volume claims for each node's local ssd. And use one PVC in the
task manager deployment. All my task manager replicas use the same
local ssd disk. But with that all my task manager replicas go to the same
node. Obviously because of the node-local volume affinity.

We can use hostpath but kubernetes recommends against it for security
reasons.

Better solution I can think of is to have statefulsets for task manager
instead of deployment and have a pvc template with the pvc backed by
dynamic external volumes like AWS EBS.

Any pointers would be appreciated. Thanks