lost partition recovery with native persistence

[Roman Novichenok <ro...@gmail.com>]

I was going over failure recovery scenarios, trying to understand logic
behind lost partitions functionality.  In the case of native persistence,
Ignite fully manages data persistence and availability.  If enough nodes in
the cluster become unavailable resulting in partitions marked lost, Ignite
keeps track of those partitions.  When nodes rejoin the cluster partitions
are automatically discovered and loaded from disk.  This can be shown by
the fact that data actually becomes available and can be retrieved using
normal get/query api's.  However, lostPartitions() lists still contain some
partitions that were previously lost and Ignite expects user to manually
mark partitions available by calling Ignite.resetLostPartitions() api.

I found some discussion about issues with topology version handling in
resetLostPartitions() in this ticket:
https://issues.apache.org/jira/browse/IGNITE-7832, but it does not address
the question, why user involvement is required at all.

Thanks,
Roman

Re: lost partition recovery with native persistence

[Павлухин Иван <vo...@gmail.com>]

Please ignore my message. I misunderstood the problem.

пт, 5 окт. 2018 г. в 14:16, Павлухин Иван <vo...@gmail.com>:

> Hi Roman,
>
> Actually, Ignite with enabled persistence supports crash recovery. It is
> mentioned in [1].
>
> [1]
> https://apacheignite.readme.io/v2.6/docs/distributed-persistent-store#section-transactional-guarantees
>
> пт, 5 окт. 2018 г. в 13:30, Maxim.Pudov <pu...@gmail.com>:
>
>> Great idea, I like it. However, it's better to discuss development plans
>> on
>> development list http://apache-ignite-developers.2346864.n4.nabble.com
>>
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>


-- 
Best regards,
Ivan Pavlukhin

Re: lost partition recovery with native persistence

[Павлухин Иван <vo...@gmail.com>]

Hi Roman,

Actually, Ignite with enabled persistence supports crash recovery. It is
mentioned in [1].

[1]
https://apacheignite.readme.io/v2.6/docs/distributed-persistent-store#section-transactional-guarantees

пт, 5 окт. 2018 г. в 13:30, Maxim.Pudov <pu...@gmail.com>:

> Great idea, I like it. However, it's better to discuss development plans on
> development list http://apache-ignite-developers.2346864.n4.nabble.com
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>


-- 
Best regards,
Ivan Pavlukhin

Re: lost partition recovery with native persistence

["Maxim.Pudov" <pu...@gmail.com>]

Great idea, I like it. However, it's better to discuss development plans on
development list http://apache-ignite-developers.2346864.n4.nabble.com




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: lost partition recovery with native persistence

[Roman Novichenok <ro...@gmail.com>]

Thanks.  I understand relying on user to determine if data is upto date
when Ignite is used as a cache.  With native persistence, Ignite is the
source of the data.  If some partitions become unavailable, there's no way
for data to become outdated.  Feels like there should be a configuration
setting to allow Ignite to auto-recover when all partitions are back online
for a cache with native persistence.

On Thu, Oct 4, 2018 at 10:54 AM Maxim.Pudov <pu...@gmail.com> wrote:

> I'm not an architect of this feature, but the explanation could be quite
> simple: data stored in lost partitions could have become outdated while
> nodes containing data were out of the cluster, so it's up to user to decide
> whether the restored data is OK, or not.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: lost partition recovery with native persistence

["Maxim.Pudov" <pu...@gmail.com>]

I'm not an architect of this feature, but the explanation could be quite
simple: data stored in lost partitions could have become outdated while
nodes containing data were out of the cluster, so it's up to user to decide
whether the restored data is OK, or not.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/