You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by fanyon <gi...@git.apache.org> on 2017/03/17 10:36:21 UTC
[GitHub] flink pull request #3561: [FLINK-6058] fix read DEFAULT_PARALLELISM from Con...
GitHub user fanyon opened a pull request:
https://github.com/apache/flink/pull/3561
[FLINK-6058] fix read DEFAULT_PARALLELISM from ContextEnvironment
fix read DEFAULT_PARALLELISM from ContextEnvironment
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fanyon/flink FLINK-6058
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3561.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3561
----
commit 446cec49e1aa75c4258c07994d0c5ae18ea0c128
Author: mengji.fy <me...@taobao.com>
Date: 2017-03-17T10:33:34Z
[FLINK-6058] fix read DEFAULT_PARALLELISM from ContextEnvironment
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3561: [FLINK-6058] fix read DEFAULT_PARALLELISM from ContextEnv...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on the issue:
https://github.com/apache/flink/pull/3561
@aljoscha Could you have a brief look at this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3561: [FLINK-6058] fix read DEFAULT_PARALLELISM from Con...
Posted by fanyon <gi...@git.apache.org>.
Github user fanyon closed the pull request at:
https://github.com/apache/flink/pull/3561
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3561: [FLINK-6058] fix read DEFAULT_PARALLELISM from ContextEnv...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/3561
Thanks for reminding, @uce \U0001f603 Already had this on my list.
The only code that reads the DEFAULT_PARALLELISM from the config should be in this piece of code in `CliFrontend`: https://github.com/apache/flink/blob/master/flink-clients/src/main/java/org/apache/flink/client/CliFrontend.java#L250. Here, we should check whether the user directly specified a parallelism and if not read the one from the config. From there the parameter is already passed to `ClusterClient.run()` where a a `ContextEnvironmentFactory` is being created. In `ContextEnvironmentFactory.createExecutionEnvironment()` we would thus have this parallelism.
The question now is whether to keep the `lastEnvCreated.setParallelism(defaultParallelism)` or pass the default parallelism to the `ContextEnvironment` so that the `StreamExecutionEnvironment` can pick it up from there in the constructor. The problem with calling `env.setParallelism()` is that the default parallelism will be "forgotten" if the user calls `env.setParallelism(PARALLELISM_DEFAULT)` where `ExecutionConfig.PARALLELISM_DEFAULT == -1`. If we manually store the default parallelism in an extra field we would keep it. This seems to be the nicer solution.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---