You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by pnowojski <gi...@git.apache.org> on 2018/03/16 12:32:33 UTC

[GitHub] flink pull request #5708: [FLINK-8984][network] Drop taskmanager.exactly-onc...

GitHub user pnowojski opened a pull request:

    https://github.com/apache/flink/pull/5708

    [FLINK-8984][network] Drop taskmanager.exactly-once.blocking.data.enabled config option

    Previously there were twe options:
    
    taskmanager.network.credit-based-flow-control.enabled
    
    and
    
    taskmanager.exactly-once.blocking.data.enabled
    
    If we disabled first one, but keept default value for the second one deadlocks will occur.
    
    By dropping taskmanager.exactly-once.blocking.data.enabled we can always use:
     - blocking BarrierBuffer for credit based flow control
     - spilling BarrierBuffer for non credit based flow control.
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): (yes / **no**)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
      - The serializers: (yes / **no** / don't know)
      - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
      - The S3 file system connector: (yes / **no** / don't know)
    
    ## Documentation
    
      - Does this pull request introduce a new feature? (yes / **no**)
      - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/pnowojski/flink f8984

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5708.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5708
    
----
commit ed992c58502779d22e3e9b5c1a120329fc20c1a6
Author: Piotr Nowojski <pi...@...>
Date:   2018-03-16T12:28:08Z

    [FLINK-8984][network] Drop taskmanager.exactly-once.blocking.data.enabled config option
    
    Previously there were twe options:
    
    taskmanager.network.credit-based-flow-control.enabled
    
    and
    
    taskmanager.exactly-once.blocking.data.enabled
    
    If we disabled first one, but keept default value for the second one deadlocks will occur.
    
    By dropping taskmanager.exactly-once.blocking.data.enabled we can always use:
     - blocking BarrierBuffer for credit based flow control
     - spilling BarrierBuffer for non credit based flow control.

----


---

[GitHub] flink issue #5708: [FLINK-8984][network] Drop taskmanager.exactly-once.block...

Posted by zhijiangW <gi...@git.apache.org>.
Github user zhijiangW commented on the issue:

    https://github.com/apache/flink/pull/5708
  
    Thanks piotr, I agree with it. 


---

[GitHub] flink issue #5708: [FLINK-8984][network] Drop taskmanager.exactly-once.block...

Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:

    https://github.com/apache/flink/pull/5708
  
    What would happen if `taskmanager.network.credit-based-flow-control.enabled` is enabled, but `taskmanager.exactly-once.blocking.data.enabled` is disabled?
    
    Is this an invalid setting?


---

[GitHub] flink pull request #5708: [FLINK-8984][network] Drop taskmanager.exactly-onc...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/flink/pull/5708


---

[GitHub] flink issue #5708: [FLINK-8984][network] Drop taskmanager.exactly-once.block...

Posted by pnowojski <gi...@git.apache.org>.
Github user pnowojski commented on the issue:

    https://github.com/apache/flink/pull/5708
  
    Yes, it's a valid setting, but has worse performance and whole credit based flow control looses it's sense. `taskmanager.exactly-once.blocking.data.enabled` setting was thought as a fallback option if there is a bug in new code path, but this is achieved by using `taskmanager.network.credit-based-flow-control.enabled` in two places (and both of those places are about using `credit-based-flow-control`).
    
    In other words, we do not think there is a much value in allowing `taskmanager.network.credit-based-flow-control.enabled` enabled, but `taskmanager.exactly-once.blocking.data.enabled` 
    disabled, and it would make the configuration more complex/confusing for the users.


---

[GitHub] flink issue #5708: [FLINK-8984][network] Drop taskmanager.exactly-once.block...

Posted by pnowojski <gi...@git.apache.org>.
Github user pnowojski commented on the issue:

    https://github.com/apache/flink/pull/5708
  
    Thanks!


---

[GitHub] flink issue #5708: [FLINK-8984][network] Drop taskmanager.exactly-once.block...

Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:

    https://github.com/apache/flink/pull/5708
  
    merging.


---