[jira] [Commented] (IGNITE-7533) Throttle writting threads according fsync progress and checkpoint writting speed instead of region fill

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-7533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339579#comment-16339579 ] 

ASF GitHub Bot commented on IGNITE-7533:
----------------------------------------

GitHub user dspavlov opened a pull request:

    https://github.com/apache/ignite/pull/3437

    IGNITE-7533: Throttle writing threads according fsync progress 

    and checkpoint write speed

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gridgain/apache-ignite ignite-7533

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/ignite/pull/3437.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3437
    
----
commit 90befa9d3bc89833eeed42a8814e8a7f51a11448
Author: dpavlov <dp...@...>
Date:   2018-01-25T18:11:07Z

    IGNITE-7533: Throttle writing threads according fsync progress and checkpoint write speed

----


> Throttle writting threads according fsync progress and checkpoint writting speed instead of region fill
> -------------------------------------------------------------------------------------------------------
>
>                 Key: IGNITE-7533
>                 URL: https://issues.apache.org/jira/browse/IGNITE-7533
>             Project: Ignite
>          Issue Type: Improvement
>          Components: persistence
>    Affects Versions: 2.3
>            Reporter: Dmitriy Pavlov
>            Assignee: Dmitriy Pavlov
>            Priority: Major
>             Fix For: 2.5
>
>         Attachments: image (2).png
>
>
> Throttling implemented under IGNITE-6334 is based on region fill percentage (ditry pages ratio) and current checkpoint progress.
> But actual progress of writting is based on write operation complete, but not on fsync() complete.
> Suppose following stage of CP is currently running: most of data is being written and fsync is started. Fsync on experiments requires more time than write, but throttling is disabled for that stage. There is enough time to unthrottled grid to fill remaining 16% of clear pages to get sufficient 75% of dirty pages and writes stops.
> Fsync progress is to be included in checkpoint progress, but actual fsync progress reported by OS is irregular both on Linux and Windows. See picture, green line is fsync progress, and yellow is write complete.
> Because fsync progress reported is not regular (the differences are 3-4 orders of magnitude) it is suggested to implement new speed based throttling policy.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)