You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by willmiao <gi...@git.apache.org> on 2015/09/08 17:19:36 UTC
[GitHub] flink pull request: [FLINK-2591]Add configuration parameter for de...
GitHub user willmiao opened a pull request:
https://github.com/apache/flink/pull/1107
[FLINK-2591]Add configuration parameter for default number of yarn containers
https://issues.apache.org/jira/browse/FLINK-2591
Add configuration parameter for default number of yarn containers
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/willmiao/flink master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1107.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1107
----
commit c1d65cae4e301a1483929b360ddcc96ad34eff31
Author: Will Miao <mi...@huawei.com>
Date: 2015-09-08T14:58:38Z
FLINK-2591
https://issues.apache.org/jira/browse/FLINK-2591
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2591]Add configuration parameter for de...
Posted by willmiao <gi...@git.apache.org>.
Github user willmiao commented on the pull request:
https://github.com/apache/flink/pull/1107#issuecomment-138603922
Thanks for your advices, I'll fix it
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2591]Add configuration parameter for de...
Posted by willmiao <gi...@git.apache.org>.
Github user willmiao closed the pull request at:
https://github.com/apache/flink/pull/1107
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2591]Add configuration parameter for de...
Posted by willmiao <gi...@git.apache.org>.
Github user willmiao commented on the pull request:
https://github.com/apache/flink/pull/1107#issuecomment-138774833
Hi rmetzger,
I've got a question: what if yarn.defaultNumberOfTaskManagers is set to a invalid value, say -2? Should users have to supply the -n argument?
I was thinking this behavior may be appropriate:
If yarn.defaultNumberOfTaskManagers < 1 (not set or invalide value): users have to supply the -n argument.
else if the user specified -n, the value of -n will have priority.
else, yarn.defaultNumberOfTaskManagers value will be used by default.
What do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2591]Add configuration parameter for de...
Posted by willmiao <gi...@git.apache.org>.
Github user willmiao commented on the pull request:
https://github.com/apache/flink/pull/1107#issuecomment-139531959
I'm sorry that something went wrong with my original repository. So, I open a new PR, and this one will be closed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2591]Add configuration parameter for de...
Posted by willmiao <gi...@git.apache.org>.
Github user willmiao commented on the pull request:
https://github.com/apache/flink/pull/1107#issuecomment-138772421
Hi rmetzger, I've got a question: what if yarn.defaultNumberOfTaskManagers is set to a invalid value, say -2? Should user have to supply the -n argument?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2591]Add configuration parameter for de...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1107#issuecomment-138885088
I agree with your proposal! If `yarn.defaultNumberOfTaskManagers` is set to an invalid value, we should make `-n` required.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2591]Add configuration parameter for de...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1107#issuecomment-138600722
Thanks a lot for opening the pull request.
Can you change the behavior of this a little bit to make the `-n` argument a required argument again? (if the config param is not set)
I think this behavior is the best:
if `yarn.numberOfTaskManagers` == -1 (default value): users have to supply the `-n` argument.
If `yarn.numberOfTaskManagers` is set to `>= 1`, the value will be used by default.
If `yarn.numberOfTaskManagers` is set to `>= 1` and the user specified `-n`, the value of `-n` will have priority.
Can you rename the configuration value to `yarn.defaultNumberOfTaskManagers`?
Can you add a test into the `flink-yarn-tests` which is using a configuration file with `yarn.defaultNumberOfTaskManagers` set to `1` WITHOUT a `-n` argument to see if everything is working as expected?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---