You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by 李 <li...@126.com> on 2022/11/29 04:22:30 UTC
How's JobManager bring up TaskManager in Application Mode or Session Mode?
Hi,
How's JobManager bring up TaskManager in Application Mode or Session Mode? I can’t get it even after reading source code of flink operator?
Any help will be appreciate, Thank you.
Mark
Re: How's JobManager bring up TaskManager in Application Mode or Session Mode?
Posted by Matthias Pohl <ma...@aiven.io.INVALID>.
Hi Mark,
the JobManager is not necessarily in charge of spinning up TaskManager
instances. It depends on the resource provider configuration you choose.
Flink differentiates between active and passive Resource Management (see
the two available implementations of ResourceManager [1]).
Active Resource Management actually takes care of spinning up new
TaskManager instances if needed (i.e. Flink runs out of free task slots).
This is handled by the corresponding AbstractResourceManageDriver
implementations [2].
In contrast, passive Resource Management (i.e. through the standalone
resource provider configurations [3]) doesn't do anything like that. Here,
Flink works with the TaskManagers that were instantiated by an external
process. Each TaskManager instance registers itself to the JobManager that
is specified in the Flink configuration which is provided to the
corresponding TaskManager instance.
I hope that helps. For future posts, please solely use the user mailing
list for questions around understanding Flink or troubleshooting. The dev
mailing list is reserved for development-related questions [4].
Matthias
[1]
https://github.com/apache/flink/blob/55a8d1a76067204e00839f1b6a2c09965434eaa4/flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManager.java#L117
[2]
https://github.com/apache/flink/blob/9815caad271a561640ffe0df7193c04270d53a25/flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/active/AbstractResourceManagerDriver.java#L33
[3]
https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/deployment/resource-providers/standalone/overview/
[4] https://flink.apache.org/community.html#mailing-lists
On Tue, Nov 29, 2022 at 5:23 AM 李 <li...@126.com> wrote:
> Hi,
>
> How's JobManager bring up TaskManager in Application Mode or Session
> Mode? I can’t get it even after reading source code of flink operator?
>
> Any help will be appreciate, Thank you.
>
> Mark
>
>
>
Re: How's JobManager bring up TaskManager in Application Mode or Session Mode?
Posted by Matthias Pohl via user <us...@flink.apache.org>.
Hi Mark,
the JobManager is not necessarily in charge of spinning up TaskManager
instances. It depends on the resource provider configuration you choose.
Flink differentiates between active and passive Resource Management (see
the two available implementations of ResourceManager [1]).
Active Resource Management actually takes care of spinning up new
TaskManager instances if needed (i.e. Flink runs out of free task slots).
This is handled by the corresponding AbstractResourceManageDriver
implementations [2].
In contrast, passive Resource Management (i.e. through the standalone
resource provider configurations [3]) doesn't do anything like that. Here,
Flink works with the TaskManagers that were instantiated by an external
process. Each TaskManager instance registers itself to the JobManager that
is specified in the Flink configuration which is provided to the
corresponding TaskManager instance.
I hope that helps. For future posts, please solely use the user mailing
list for questions around understanding Flink or troubleshooting. The dev
mailing list is reserved for development-related questions [4].
Matthias
[1]
https://github.com/apache/flink/blob/55a8d1a76067204e00839f1b6a2c09965434eaa4/flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManager.java#L117
[2]
https://github.com/apache/flink/blob/9815caad271a561640ffe0df7193c04270d53a25/flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/active/AbstractResourceManagerDriver.java#L33
[3]
https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/deployment/resource-providers/standalone/overview/
[4] https://flink.apache.org/community.html#mailing-lists
On Tue, Nov 29, 2022 at 5:23 AM 李 <li...@126.com> wrote:
> Hi,
>
> How's JobManager bring up TaskManager in Application Mode or Session
> Mode? I can’t get it even after reading source code of flink operator?
>
> Any help will be appreciate, Thank you.
>
> Mark
>
>
>