You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by tillrohrmann <gi...@git.apache.org> on 2016/11/17 16:45:56 UTC

[GitHub] flink pull request #2826: [FLINK-5085] Execute CheckpointCoordinator's state...

GitHub user tillrohrmann opened a pull request:

    https://github.com/apache/flink/pull/2826

    [FLINK-5085] Execute CheckpointCoordinator's state discard calls asynchronously

    This PR is a back port of #2825 for the release 1.1 branch. It is based on #2816. Thus only a70097d is relevant.
    
    The `CheckpointCoordinator` is now given an `Executor` which is used to execute the state discard
    calls asynchronously. This will prevent blocking operations to be executed from within the
    calling thread. The provided `Executor` is the same executor as the one used for the cleanup in the `ZooKeeperStateHandleStore`.
    
    The executors are now gracefully shutdown after the `JobManager` has terminated. If the executors don't shut down in the given time (akka ask timeout), then the executors are shut down hard.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tillrohrmann/flink backportMakeCheckpointCoordinatorNotBlocking

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/2826.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2826
    
----
commit 357690b359a2890ec1842a20d345675b79d61cd1
Author: Till Rohrmann <tr...@apache.org>
Date:   2016-11-15T21:45:04Z

    [FLINK-5073] Use Executor to run ZooKeeper callbacks in ZooKeeperStateHandleStore
    
    Use dedicated Executor to run ZooKeeper callbacks in ZooKeeperStateHandleStore instead
    of running it in the ZooKeeper client's thread. The callback can be blocking because it
    discards state which might entail deleting files from disk.
    
    Add TestExecutors

commit 640bfef9a176d57fa70d8ac21b8675897fae11ec
Author: Till Rohrmann <tr...@apache.org>
Date:   2016-11-16T17:33:54Z

    [FLINK-5082] Pull ExecutorService lifecycle management out of the JobManager
    
    The provided ExecutorService will no longer be closed by the JobManager. Instead the
    lifecycle is managed outside of it where it was created. This will give a nicer behaviour,
    because it better seperates responsibilities.

commit 9de05526e49158a5bde1342afe602f358cae993f
Author: Till Rohrmann <tr...@apache.org>
Date:   2016-11-16T17:51:05Z

    Introduce dedicated Executor for blocking io operations

commit a70097d4ac619f9203604f6991d293a7b0f55b54
Author: Till Rohrmann <tr...@apache.org>
Date:   2016-11-17T14:39:11Z

    [FLINK-5085] Execute CheckpointCoordinator's state discard calls asynchronously
    
    The CheckpointCoordinator is now given an Executor which is used to execute the state discard
    calls asynchronously. This will prevent blocking operations to be executed from within the
    calling thread.
    
    Shut down ExecutorServices gracefully

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #2826: [backport] [FLINK-5085] Execute CheckpointCoordinator's s...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:

    https://github.com/apache/flink/pull/2826
  
    Review @StefanRRichter 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2826: [backport] [FLINK-5085] Execute CheckpointCoordina...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann closed the pull request at:

    https://github.com/apache/flink/pull/2826


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #2826: [backport] [FLINK-5085] Execute CheckpointCoordinator's s...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:

    https://github.com/apache/flink/pull/2826
  
    Rebased the PR on the latest release-1.1 branch. 
    
    Review @uce, @StephanEwen if you have time.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #2826: [backport] [FLINK-5085] Execute CheckpointCoordinator's s...

Posted by StefanRRichter <gi...@git.apache.org>.
Github user StefanRRichter commented on the issue:

    https://github.com/apache/flink/pull/2826
  
    +1 LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #2826: [backport] [FLINK-5085] Execute CheckpointCoordinator's s...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:

    https://github.com/apache/flink/pull/2826
  
    Build passed locally: https://travis-ci.org/tillrohrmann/flink/builds/177725509. Merging the PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #2826: [backport] [FLINK-5085] Execute CheckpointCoordinator's s...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:

    https://github.com/apache/flink/pull/2826
  
    Thanks for the review @StefanRRichter. Once Travis gives green light, I'll merge the PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---