You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by bbayani <gi...@git.apache.org> on 2017/08/30 10:23:28 UTC
[GitHub] flink pull request #4622: [FLINK-7294]:[flink-mesos] mesos.resourcemanager.f...
GitHub user bbayani opened a pull request:
https://github.com/apache/flink/pull/4622
[FLINK-7294]:[flink-mesos] mesos.resourcemanager.framework.role not working
Jira Issue: FLINK-7294
## What is the purpose of the change
This pull request uses role set in mesos.resourcemanager.framework.role and applies it for resources such as CPU, mem, ports. Due to this framework considers resource offers coming from mesos-agents with specified role and is able to spawn up task-managers on mesos-agent running with specific role that role *.
## Brief change log
- Updated Utils.java to take in role information for constructing scalar / ranges resource values.
- Updated LaunchableMesosWorker to use framework role set in config.
- Updated tests in LaunchCoordinatorTest.scala to pass role argument.
## Verifying this change
Part of change is already covered by existing tests, such as LaunchCoordinatorTest.scala.
Also, manually verified the change by running a flink-mesos cluster with 1 job-manager and 3 task-managers. The flink was deployed on a mesos-cluster where mesos-workers were running with specific role and not role '*'.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: yes, it affects deployment on Mesos.
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not documented
@EronWright : PTAL. Thanks!
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bbayani/flink mesos_role_issue
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4622.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4622
----
commit 883c3866302b8073b60403b65c3aac85759b891c
Author: bbayani <bb...@cisco.com>
Date: 2017-08-30T09:54:54Z
[FLINK-7294]:mesos.resourcemanager.framework.role not working
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4622: [FLINK-7294]:[flink-mesos] mesos.resourcemanager.framewor...
Posted by EronWright <gi...@git.apache.org>.
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4622
@tillrohrmann please take a look and merge if you're OK with it.
---
[GitHub] flink issue #4622: [FLINK-7294]:[flink-mesos] mesos.resourcemanager.framewor...
Posted by bbayani <gi...@git.apache.org>.
Github user bbayani commented on the issue:
https://github.com/apache/flink/pull/4622
@EronWright Yes, I have verified the case when mesos.resourcemanager.framework.role is not configured in flink-conf.yaml (and thus defaults to *). In this case, it launched all the tasks only on worker running with role *.
---
[GitHub] flink issue #4622: [FLINK-7294]:[flink-mesos] mesos.resourcemanager.framewor...
Posted by EronWright <gi...@git.apache.org>.
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4622
@bbayani thanks very much for this contribution. Please confirm my understanding of the issue and verification. It appears that the resources that are requested via `TaskInfo` must reference the framework role, otherwise the resources are actually drawn from the unreserved pool. Your cluster is configured such that all resources are reserved for a certain role, and so Flink is unable to launch without this fix despite being configured to use the role.
Have you verified correct behavior in the case where Flink's role is not configured (and there is unreserved capacity)? Is the `TaskInfo` role set to `*` and is that correct?
---
[GitHub] flink pull request #4622: [FLINK-7294]:[flink-mesos] mesos.resourcemanager.f...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4622
---
[GitHub] flink issue #4622: [FLINK-7294]:[flink-mesos] mesos.resourcemanager.framewor...
Posted by bbayani <gi...@git.apache.org>.
Github user bbayani commented on the issue:
https://github.com/apache/flink/pull/4622
Can flink-mesos contributors/owners please take a look at this PR?
---
[GitHub] flink issue #4622: [FLINK-7294]:[flink-mesos] mesos.resourcemanager.framewor...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4622
Thanks for the contribution @bbayani and the review @EronWright. Changes look good to me. Will merge the fix and backport it to the `1.3` release branch.
---