You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Sergio Sainz (Jira)" <ji...@apache.org> on 2023/04/13 14:23:00 UTC
[jira] [Updated] (FLINK-31796) Support service mesh istio with Flink kubernetes (both native and operator) for secure communications
[ https://issues.apache.org/jira/browse/FLINK-31796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sergio Sainz updated FLINK-31796:
---------------------------------
Description:
Currently Flink Native Kubernetes does not support istio + TLS cleanly : Flink assumes that pods will be able to communicate by ip-address, meanwhile istio + TLS does not allow routing by pod's ip-address.
This ticket is to track effort to support Flink with istio. Some workaround is to disable the istio sidecar container, but this is not secure [https://doc.akka.io/docs/akka-management/current/bootstrap/istio.html]. Akka allows to secure the channel manually, but there is no documentation how to do this in the context of Flink. One potential solution for this is to have the documentation about how to configure akka cluster + TLS in flink.
For example when using native kubernetes deployment mode with high-availability (HA), and when new TaskManager pod is started to process a job, the TaskManager pod will attempt to register itself to the resource manager (JobManager). the TaskManager looks up the resource manager per ip-address (akka.tcp://flink@192.168.140.164:6123/user/rpc/resourcemanager_1).
Other affected features is metric collection.
Please see FLINK-31775 and FLINK-28171. Especially comment "{_}Flink currently just doesn't support Istio.{_}"
was:
Currently Flink Native Kubernetes does not support istio + TLS cleanly : Flink assumes that pods will be able to communicate by ip-address, meanwhile istio + TLS does not allow routing by pod's ip-address.
This ticket is to track effort to support Flink with istio. Some workaround is to disable the istio sidecar container, but this is not secure [https://doc.akka.io/docs/akka-management/current/bootstrap/istio.html]. Akka allows to secure the channel manually, but there is no documentation how to do this in the context of Flink. One potential solution for this is to have the documentation about how to configure akka cluster + TLS in flink.
For example when using native kubernetes deployment mode with high-availability (HA), and when new TaskManager pod is started to process a job, the TaskManager pod will attempt to register itself to the resource manager (JobManager). the TaskManager looks up the resource manager per ip-address (akka.tcp://flink@192.168.140.164:6123/user/rpc/resourcemanager_1).
Other affected features is metric collection.
Please see FLINK-31775 and FLINK-28171. Especially comment "{_}Flink currently just doesn't support Istio. If anything, it's a new feature/improvement.{_} "
> Support service mesh istio with Flink kubernetes (both native and operator) for secure communications
> -----------------------------------------------------------------------------------------------------
>
> Key: FLINK-31796
> URL: https://issues.apache.org/jira/browse/FLINK-31796
> Project: Flink
> Issue Type: New Feature
> Components: Deployment / Kubernetes
> Affects Versions: 1.17.0
> Reporter: Sergio Sainz
> Priority: Major
>
> Currently Flink Native Kubernetes does not support istio + TLS cleanly : Flink assumes that pods will be able to communicate by ip-address, meanwhile istio + TLS does not allow routing by pod's ip-address.
> This ticket is to track effort to support Flink with istio. Some workaround is to disable the istio sidecar container, but this is not secure [https://doc.akka.io/docs/akka-management/current/bootstrap/istio.html]. Akka allows to secure the channel manually, but there is no documentation how to do this in the context of Flink. One potential solution for this is to have the documentation about how to configure akka cluster + TLS in flink.
> For example when using native kubernetes deployment mode with high-availability (HA), and when new TaskManager pod is started to process a job, the TaskManager pod will attempt to register itself to the resource manager (JobManager). the TaskManager looks up the resource manager per ip-address (akka.tcp://flink@192.168.140.164:6123/user/rpc/resourcemanager_1).
> Other affected features is metric collection.
> Please see FLINK-31775 and FLINK-28171. Especially comment "{_}Flink currently just doesn't support Istio.{_}"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)