You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/04/06 13:04:00 UTC
[jira] [Commented] (IGNITE-8017) Disable WAL during initial preloading

    [ https://issues.apache.org/jira/browse/IGNITE-8017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16428277#comment-16428277 ] 

ASF GitHub Bot commented on IGNITE-8017:
----------------------------------------

GitHub user ilantukh opened a pull request:

    https://github.com/apache/ignite/pull/3770

    IGNITE-8017 (2)

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gridgain/apache-ignite ignite-8017-8122

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/ignite/pull/3770.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3770
    
----
commit 604ee719b304d0b4cf4caabaa6fa16b5a980e04e
Author: Pavel Kovalenko <jo...@...>
Date:   2018-04-04T09:33:10Z

    IGNITE-8122 Restore partition state to OWNING if unable to read from page memory.

commit 67a48943b876b193067625fb564ab44d57a83f56
Author: Ilya Lantukh <il...@...>
Date:   2018-03-23T13:10:41Z

    ignite-8017 : Simple test.

commit 0021e39fe4a232a402c93aa6431048cdd00ca74c
Author: Ilya Lantukh <il...@...>
Date:   2018-03-23T13:46:29Z

    ignite-8017 : Basic implementation.

commit 3d446aa7bf78d53511725eb33e2546293e641167
Author: Ilya Lantukh <il...@...>
Date:   2018-03-23T14:05:55Z

    ignite-8017 : Configuration property.

commit bda2ba4e25e6699974520cc378638be22f4e1db3
Author: Ilya Lantukh <il...@...>
Date:   2018-03-23T14:09:04Z

    ignite-8017 : Property value check.

commit 9f50edc7a68dbf3253ec19ef15a8530d8484377a
Author: Ilya Lantukh <il...@...>
Date:   2018-03-23T14:46:19Z

    ignite-8017 : Fixed test finalization.

commit da2d400f578cd57c129c9a410e48b4586310a728
Author: Ilya Lantukh <il...@...>
Date:   2018-03-26T12:34:14Z

    ignite-8017 : Fixed parameter handling.

commit 34aa645d93c67ab7cea75edd4357339aab785bc8
Author: Ilya Lantukh <il...@...>
Date:   2018-03-26T16:25:00Z

    ignite-8017 : WIP.

commit 620b15b97511ade697ed7ec3255f4312afe857c5
Author: Ilya Lantukh <il...@...>
Date:   2018-03-27T15:04:03Z

    ignite-8017 : Test refactoring.

commit adab75755da9dcdec82d27e8d4d4099491c6d2f3
Author: Ilya Lantukh <il...@...>
Date:   2018-03-27T15:19:26Z

    ignite-8017 : Test for local and global WAL state.

commit 95492f025381521b76b480d4871a88a5fc9e3b40
Author: Ilya Lantukh <il...@...>
Date:   2018-03-27T15:27:15Z

    ignite-8017 : Separate handling for global and local WAL state.

commit 6da9bd39a28a6aee21613c2fb660ad6a9c6b62fb
Author: Ilya Lantukh <il...@...>
Date:   2018-03-29T13:58:58Z

    ignite-8017 : WIP.

commit 66968f98a3579c939ea0166008c36f15dbd0d484
Author: Ilya Lantukh <il...@...>
Date:   2018-04-03T13:43:07Z

    ignite-8017 : WIP.

commit d008fde6ded45d3f31ca3848bbbe347d75a05f30
Author: Ilya Lantukh <il...@...>
Date:   2018-04-03T14:04:32Z

    ignite-8017 : WIP.

commit 06d820950d87c0c4405c5b063596edcde37b0bf3
Author: Ilya Lantukh <il...@...>
Date:   2018-04-04T16:01:55Z

    ignite-8017 : WIP.

commit 4d8b7bdd0724da39d4cc45b403624c479984d1f8
Author: Ilya Lantukh <il...@...>
Date:   2018-04-04T16:06:31Z

    ignite-8017 : WIP.

commit 5022a554aa2ac73200f9180314f2ef69b54afb38
Author: Ilya Lantukh <il...@...>
Date:   2018-04-05T11:54:12Z

    ignite-8017 : WIP.

commit c43c598916bd9bda2f624a214cefcc28b0380a5c
Author: Pavel Kovalenko <jo...@...>
Date:   2018-04-05T14:26:53Z

    IGNITE-8122 Restore partitions when persistence is enabled with OWNING default state.

commit a061cdba3a46f5384910fecf89366101f904ccdf
Author: Pavel Kovalenko <jo...@...>
Date:   2018-04-05T14:50:20Z

    IGNITE-8122 Move OWN logic to GridDhtLocalPartition constructor.

commit 9259407a462633a68de6dcd7d3ae135c8c7c0b37
Author: Pavel Kovalenko <jo...@...>
Date:   2018-04-05T17:15:39Z

    IGNITE-8122 Docs.

commit 20979dc4874cfeca649b29d8827d522f3e57ee67
Author: Pavel Kovalenko <jo...@...>
Date:   2018-04-05T17:16:54Z

    Merge branch 'master' into ignite-8122

commit 96ac09a87690fbe8a0d9623b01e12715905c568a
Author: Ilya Lantukh <il...@...>
Date:   2018-04-06T12:28:05Z

    Merge branch 'ignite-8017' of https://github.com/gridgain/apache-ignite into ignite-8017-8122

commit fb079afe7d96aa0c5511d58c7a9db3b77ee8813f
Author: Ilya Lantukh <il...@...>
Date:   2018-04-06T13:02:16Z

    ignite-8017 : WIP.

----


> Disable WAL during initial preloading
> -------------------------------------
>
>                 Key: IGNITE-8017
>                 URL: https://issues.apache.org/jira/browse/IGNITE-8017
>             Project: Ignite
>          Issue Type: Improvement
>            Reporter: Ilya Lantukh
>            Assignee: Ilya Lantukh
>            Priority: Major
>              Labels: iep-16
>             Fix For: 2.5
>
>
> While handling SupplyMessage, node handles each supplied data entry separately, which causes a WAL record for each entry to be written. It significantly limits preloading speed.
> We can improve rebalancing speed and reduce pressure on disk by disabling WAL until all data is loaded. The disadvantage of this approach is that data might get corrupted if node crashes - but node that crashed during preloading has to clear all it's data anyway. However, it is important to distinguish situations when new node joined cluster or added to baseline topology (and doesn't hold any data) and when additional partitions got assigned to node after baseline topology changed (in this case node has to keep all data in consistent state).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)