[jira] [Commented] (FLINK-8674) Efficiently handle alwaysFlush case (0ms flushTimeout)

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369165#comment-16369165 ] 

ASF GitHub Bot commented on FLINK-8674:
---------------------------------------

GitHub user pnowojski opened a pull request:

    https://github.com/apache/flink/pull/5526

    [FLINK-8674][runtime] Improve performance of flushAlways in StreamRecordWriter

    This PR reduces the number of data notifications in case of `flushAlways = true`. Instead of notifying all of the channels/subpartitions, notify only the one that has just been written to.
    
    ## Verifying this change
    
    This change is already covered by existing tests.
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): (yes / **no**)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
      - The serializers: (yes / **no** / don't know)
      - The runtime per-record code paths (performance sensitive): (**yes** / no / don't know)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
      - The S3 file system connector: (yes / **no** / don't know)
    
    ## Documentation
    
      - Does this pull request introduce a new feature? (yes / **no**)
      - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/pnowojski/flink f8674

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5526.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5526
    
----
commit 56eb9aa6a3ee6167e4e0cf7d22cbf81ed2d11ccf
Author: Piotr Nowojski <pi...@...>
Date:   2018-02-15T13:41:39Z

    [FLINK-8674][runtime] Improve performance of flushAlways in StreamRecordWriter

----


> Efficiently handle alwaysFlush case (0ms flushTimeout)
> ------------------------------------------------------
>
>                 Key: FLINK-8674
>                 URL: https://issues.apache.org/jira/browse/FLINK-8674
>             Project: Flink
>          Issue Type: Sub-task
>            Reporter: Piotr Nowojski
>            Assignee: Piotr Nowojski
>            Priority: Major
>
> Changes in data notifications introduced alongside BufferConsumer significantly degraded performance of 0ms flushTimeout.
>  
> Previously flushing after writing only record, effectively triggered only one data notification for the sub-partition/channel to which this record was written. With low latency improvements, this changed and flush is now triggering data notifications for all of the partitions. This can (and should) be easily fixed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)