You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by senorcarbone <gi...@git.apache.org> on 2016/01/20 14:34:52 UTC

[GitHub] flink pull request: [FLINK-3256] Fix colocation group re-instantia...

GitHub user senorcarbone opened a pull request:

    https://github.com/apache/flink/pull/1526

    [FLINK-3256] Fix colocation group re-instantiation

    This PR deals with the problem of inconsistent colocation groups upon reconfiguration. The problem was that we were removing shared constraints multiple times for each ExecutionJobVertex, thus, colocated vertices, in the same co-location group, ended up being scheduled with different constraints leading to wrong redeployment.
    
    To deal with it we keep all distinct colocation groups in the execution graph and reset them once outside the individual ExecutionJobVertex re-instantiation. There is also a new test that is used to check whether certain properties are consistent after reconfiguration. We can potentially add more properties in the same test to ensure that they are also maintained upon reconfiguration.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/senorcarbone/flink egfix

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/1526.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1526
    
----
commit a8f24b5003885596f48eaa73b24b94dfbc5380e6
Author: Paris Carbone <pa...@kth.se>
Date:   2016-01-20T02:03:41Z

    [FLINK-3256] Fix colocation group re-instantiation

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-3256] Fix colocation group re-instantia...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/flink/pull/1526


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-3256] Fix colocation group re-instantia...

Posted by senorcarbone <gi...@git.apache.org>.
Github user senorcarbone commented on the pull request:

    https://github.com/apache/flink/pull/1526#issuecomment-173575376
  
    I removed the groups from the ExecutionGraph. Hope it's good now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-3256] Fix colocation group re-instantia...

Posted by senorcarbone <gi...@git.apache.org>.
Github user senorcarbone commented on the pull request:

    https://github.com/apache/flink/pull/1526#issuecomment-173260776
  
    Sure, I will do it on the fly. Thanks for checking it!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-3256] Fix colocation group re-instantia...

Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:

    https://github.com/apache/flink/pull/1526#issuecomment-174521559
  
    Looks good, +1 to merge!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-3256] Fix colocation group re-instantia...

Posted by senorcarbone <gi...@git.apache.org>.
Github user senorcarbone commented on the pull request:

    https://github.com/apache/flink/pull/1526#issuecomment-174486113
  
    I guess this fix ok to merge. Any objections?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-3256] Fix colocation group re-instantia...

Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:

    https://github.com/apache/flink/pull/1526#issuecomment-173240931
  
    Thanks for fixing this, looks like a good issue!
    
    One suggestion for a change: Rather than collecting the CoLocationGroups in an extra set in the ExecutionGraph, can you iterate over the ExecutionJobvertices and simply reset them there?
    
    I think each field less on the ExecutionGraph makes this simpler to maintain and more future-proof.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---