[jira] [Created] (FLINK-31796) Support service mesh istio with Flink kubernetes (both native and operator) for secure communications

["Sergio Sainz (Jira)" <ji...@apache.org>]

Sergio Sainz created FLINK-31796:
------------------------------------

             Summary: Support service mesh istio with Flink kubernetes (both native and operator) for secure communications
                 Key: FLINK-31796
                 URL: https://issues.apache.org/jira/browse/FLINK-31796
             Project: Flink
          Issue Type: New Feature
          Components: Deployment / Kubernetes
    Affects Versions: 1.17.0
            Reporter: Sergio Sainz


Currently Flink Native Kubernetes does not support istio + TLS cleanly : Flink assumes that pods will be able to communicate by ip-address, meanwhile istio + TLS does not allow routing by pod's ip-address.

This ticket is to track effort to support Flink with istio. Some workaround is to disable the istio sidecar container, but this is not secure [https://doc.akka.io/docs/akka-management/current/bootstrap/istio.html]. Akka allows to  secure the channel manually, but there is no documentation how to do this in the context of Flink. One potential solution for this is to have the documentation about how to configure akka cluster + TLS in flink.

For example when using native kubernetes deployment mode with high-availability (HA), and when new TaskManager pod is started to process a job, the TaskManager pod will attempt to register itself to the resource manager (JobManager). the TaskManager looks up the resource manager per ip-address (akka.tcp://flink@192.168.140.164:6123/user/rpc/resourcemanager_1).

Other affected features is metric collection.

Please see FLINK-31775 and FLINK-28171. Especially comment "{_}Flink currently just doesn't support Istio. If anything, it's a new feature/improvement.{_} "



--
This message was sent by Atlassian Jira
(v8.20.10#820010)