[jira] [Commented] (FLINK-25764) Docker sets JobManager's rpc address to same host by default

["Chesnay Schepler (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-25764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480927#comment-17480927 ] 

Chesnay Schepler commented on FLINK-25764:
------------------------------------------

The docker image having a default for JOB_MANAGER_RPC_ADDRESS is not a problem in that sense. If we were to remove that you'd still have a default "localhost" setting through the Flink config.
So as I understand it all it does is improve the out-of-the-box experience if you're just experimenting with docker.

??(b) never set a default setting for any non-JobManager container??

We can't, or rather shouldn't, make this distinction. The "jobmanager"/"taskmanager"/etc branches you see in the docker image are essentially legacy code that we'd ideally get rid of, so that users use the distribution in the docker image as if it weren't a docker image (i.e., they use it as if they'd just use it without docker).


The core issue is that the docker scripts expect all changes to the config to happen _after_ it sets the defaults (e.g., by setting them via FLINK_PROPERTIES or as dynamic properties when starting the JM/TM.)

What we need is a way to detect whether the flink config that is in the container is actually the default config, or already modified by user. If it is the latter, we wouldn't change anything. I'm not sure what the best option is for that, but maybe a simple checksum would work.


> Docker sets JobManager's rpc address to same host by default
> ------------------------------------------------------------
>
>                 Key: FLINK-25764
>                 URL: https://issues.apache.org/jira/browse/FLINK-25764
>             Project: Flink
>          Issue Type: Bug
>          Components: Deployment / Kubernetes, flink-docker
>    Affects Versions: 1.15.0
>            Reporter: Niklas Semmler
>            Priority: Minor
>              Labels: usability
>
> In the [docker entrypoint|https://github.com/apache/flink-docker/blob/master/1.14/scala_2.12-java8-debian/docker-entrypoint.sh], the JOB_MANAGER_RPC_ADDRESS is set to the current host by default (line 25). This environment variable overrides the value set for jobmanager.rpc.address in the flink config (line 78, 71). For the TaskManager, this means that it tries to find the JobManager on the same host. When this is not the case, the TaskManager will retry and ultimately terminate. Hence, for cluster deployments, the variable has to be defined when starting the docker.
> For Kubernetes deployments, the TaskManager cannot connect to the jobmanager.rpc.address even when it is defined by the flink configmap. However, we don't see this problem pop up, because for now the configmap is mounted read-only into the containers (see FLINK-21383 for more details).
> To simplify this configuration, I propose to (a) never set a default setting for JOB_MANAGER_RPC_ADDRESS at all or (b) never set a default setting for any non-JobManager container. The only down-side is that all docker deployments will have to define JOB_MANAGER_RPC_ADDRESS, even when TaskManager and JobManager run on the same node.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)