[jira] [Commented] (FLINK-21036) Consider removing automatic configuration fo number of slots from docker

["Yang Wang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17268348#comment-17268348 ] 

Yang Wang commented on FLINK-21036:
-----------------------------------

I am in favor of removing the automatic configuration for {{taskmanager.numberOfTaskSlots}}. Because when we set the cpu limit for docker, we will get an incorrect number.
{code:java}
$sudo docker run --cpus 1 -it apache/flink:1.12.1 bash -c 'echo $(grep -c ^processor /proc/cpuinfo)'
96
{code}
Also for standalone Flink on Kubernetes use case, almost all the users are mounting a ConfigMap[1] for the {{flink-conf.yaml}}. And the {{docker-entrypoint.sh}} do not have the permission to override the conf based the processor. The mounted {{flink-conf.yaml}} from ConfigMap is read-only. I believe we do not expect it to be overridden as well.

 

[1]. https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/kubernetes.html#common-cluster-resource-definitions

> Consider removing automatic configuration fo number of slots from docker
> ------------------------------------------------------------------------
>
>                 Key: FLINK-21036
>                 URL: https://issues.apache.org/jira/browse/FLINK-21036
>             Project: Flink
>          Issue Type: Improvement
>          Components: Deployment / Scripts
>            Reporter: Chesnay Schepler
>            Priority: Major
>             Fix For: 1.13.0
>
>
> The {{docker-entrypoint.sh}} supports setting the number of task slots via the {{TASK_MANAGER_NUMBER_OF_TASK_SLOTS}} environment variable, which defaults to the number of cpu cores via {{$(grep -c ^processor /proc/cpuinfo)}}.
> The environment variable itself is redundant nowadays since we introduced {{FLINK_PROPERTIES}}, and is no longer documented.
> Defaulting to the number of CPU cores can be considered convenience, but it seems odd to have this specific to docker while the distribution defaults to {{1}}.
> The bigger issue in my mind though is that this creates a configuration mismatch between the Job- and TaskManager processes; the ResourceManager specifically needs to know how many slots a worker has to make decisions about redundancy and allocating resources.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)