You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Till Rohrmann (JIRA)" <ji...@apache.org> on 2018/02/20 10:04:02 UTC
[jira] [Reopened] (FLINK-8711) Flink with YARN uses wrong
SlotsPerTaskManager
[ https://issues.apache.org/jira/browse/FLINK-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Till Rohrmann reopened FLINK-8711:
----------------------------------
There is actually a problem: Due to this "optimization", we basically don't allow the execution of jobs where we have multiple slot sharing groups. The number of maximum slots per TM will be determined by simply considering the user specified parallelism which is not necessarily the number of required slots. Thus, it is impossible for a user to execute these kind of jobs on a Yarn cluster. The same actually applies to the {{LocalExecutor}}.
> Flink with YARN uses wrong SlotsPerTaskManager
> ----------------------------------------------
>
> Key: FLINK-8711
> URL: https://issues.apache.org/jira/browse/FLINK-8711
> Project: Flink
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.3.2
> Reporter: Aleksandr
> Priority: Major
>
> I see wrong behavior for Flink in YARN.
> I tried to setup SlotsPerTaskManager using "-ys 2 ", but it used only 1 slot.
> I found the code [https://github.com/apache/flink/blob/master/flink-yarn/src/main/java/org/apache/flink/yarn/cli/FlinkYarnSessionCli.java#L387]
> For example, when I have :"-yn 7 -ys 2 -p 2" in log I see:
> "The YARN cluster has 14 slots available, but the user requested a parallelism of 2 on YARN. Each of the 7 TaskManagers will get 1 slots."
> Why can't we use -ys with -p?
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)