You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/07/06 14:59:00 UTC
[jira] [Commented] (FLINK-7113) Make ClusterDescriptor independent
of Flink cluster size
[ https://issues.apache.org/jira/browse/FLINK-7113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076649#comment-16076649 ]
ASF GitHub Bot commented on FLINK-7113:
---------------------------------------
GitHub user tillrohrmann opened a pull request:
https://github.com/apache/flink/pull/4271
[FLINK-7113] Make ClusterDescriptor independent of cluster size
This PR is based on #4270.
In order to pull some of the cluster deployment logic out of the `AbstractYarnClusterDescriptor`, this PR introduces a `ClusterSpecification`. This specification contains information about the size of the cluster to be started. Instead of setting these values via setters directly on the `ClusterDescriptor`, we provide the information at deployment time via the `ClusterSpecification`.
The way I envision the `ClusterDescriptor` to work, is to only contain information which is necessary to talk to a resource manager, such as Mesos or Yarn. All Flink cluster specific information should be provided to the `deploy` methods. This also allows us better share code between different `ClusterDescriptor`, e.g. Mesos and Yarn.
This is only the start of a refactoring of the `ClusterDescriptor`. Much more information can be pulled out of the `AbstractYarnClusterDescriptor`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tillrohrmann/flink refactorClusterDescriptor
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4271.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4271
----
commit 9cfb9718d7b64e253aaf636dc17b8939e2592a98
Author: Till Rohrmann <tr...@apache.org>
Date: 2017-07-06T07:33:18Z
[FLINK-7110] [client] Add per-job cluster deployment interface
commit 0143d7e33b7d99ce8371e455458b730ba2c13d06
Author: Till Rohrmann <tr...@apache.org>
Date: 2017-07-06T12:00:21Z
[FLINK-7113] Make ClusterDescriptor independent of cluster size
The deploySession method now is given a ClusterSpecification which specifies the
size of the cluster which it is supposed to deploy.
----
> Make ClusterDescriptor independent of Flink cluster size
> --------------------------------------------------------
>
> Key: FLINK-7113
> URL: https://issues.apache.org/jira/browse/FLINK-7113
> Project: Flink
> Issue Type: Sub-task
> Components: Cluster Management
> Affects Versions: 1.4.0
> Reporter: Till Rohrmann
> Assignee: Till Rohrmann
>
> The {{ClusterDescriptor}} needs to know the size of the Flink cluster it is supposed to deploy. As a consequence we have the {{AbstractYarnClusterDescriptor}} which is configured with this information via setters. I think it would be better to give the cluster size to the {{ClusterDescriptor}} via the {{deploySession(ClusterSpecification)}} call. That way we better decouple the deployment from the cluster configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)