You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by sachingoel0101 <gi...@git.apache.org> on 2015/09/03 13:39:23 UTC
[GitHub] flink pull request: [FLINK-2615]Preserve executors for the entire ...
GitHub user sachingoel0101 opened a pull request:
https://github.com/apache/flink/pull/1088
[FLINK-2615]Preserve executors for the entire run of program.
1. Prevents re-starts of local cluster several times on multiple executions.
2. Preserves the executor states for the entire run of the program.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sachingoel0101/flink executor_fixes
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1088.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1088
----
commit 0a0b9e8b6630c5418a248b13c0b4e71e21b07804
Author: Sachin Goel <sa...@gmail.com>
Date: 2015-09-02T17:34:41Z
Preserve executors for the entire run of program.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2615]Preserve executors for the entire ...
Posted by sachingoel0101 <gi...@git.apache.org>.
Github user sachingoel0101 commented on the pull request:
https://github.com/apache/flink/pull/1088#issuecomment-137421268
Sure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2615]Preserve executors for the entire ...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on the pull request:
https://github.com/apache/flink/pull/1088#issuecomment-137421164
I would move the discussion to the respective PR. Sachin, could you close this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2615]Preserve executors for the entire ...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/1088#issuecomment-137418968
The current behavior is actually desired. You can explicitly start the local executor, and explicitly stop it. When you did not explicitly start it, you get an auto-shut down executor. This is quite flexible, no?
The LocalExecutionEnvironment is a local one-time environment. There is code to extend it to sessions, which is really what we want to "keep context and data around".
For that, there is part of the work here: https://github.com/apache/flink/pull/858
It follows a very principled approach, where the responsibility of deciding clearing / disposal of cached context and data is bound to a session. Sessions have timeouts and scope reapers, and are managed by the execution environments, not the executors.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2615]Preserve executors for the entire ...
Posted by sachingoel0101 <gi...@git.apache.org>.
Github user sachingoel0101 closed the pull request at:
https://github.com/apache/flink/pull/1088
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2615]Preserve executors for the entire ...
Posted by sachingoel0101 <gi...@git.apache.org>.
Github user sachingoel0101 commented on the pull request:
https://github.com/apache/flink/pull/1088#issuecomment-137420928
Aha. I just had a brief look at the PR.
I had initially thought of implementing a start and stop method in the `ExecutionEnvironment` but decided against it, since it changes the api for very minor changes.
Session management is a way better solution. Any timeline on when those changes could get merged?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---