Data loss validation. TopologyValidator

[Raja <ye...@gmail.com>]

Hi,

Whats the best way to check to see if the cache data is lost (partially) ? I
was looking at 'Topology Validator'  as an option.

Scenario : If there any multiple node failures occurred at the same time and
ignite loses primary and as well as backup copies of partitions and how to
know if this happened and data is lost? 

Once you detect the data loss, how to restrict/reject clients and because
the cache is partial and not complete.

Appreciate your inputs how to handle such situations in Ignite.

Thanking you
Raja





--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-loss-validation-TopologyValidator-tp15147.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data loss validation. TopologyValidator

[ezhuravlev <e....@gmail.com>]

TopologyValidator blocks cache write operations on this topology and it can
be unblocked only after new topology changes. 

Looks like Ignite doesn't have what you want out of the box and you need to
impelement it yourself. For example, you can introduce flag
isClusterAccessible, which will have default value "true" and will be set to
"false" when you received EVT_CACHE_REBALANCE_PART_DATA_LOST event



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-loss-validation-TopologyValidator-tp15147p16026.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data loss validation. TopologyValidator

[Raja <ye...@gmail.com>]

Appreciate if anyone has thoughts on how to solve the above issue



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-loss-validation-TopologyValidator-tp15147p15958.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data loss validation. TopologyValidator

[Raja <ye...@gmail.com>]

/Scenario : 

If there are multiple node failures occurred at the same time and ignite
loses primary and as well as backup copies of partitions and how to know if
this happened and data is lost? 

Once you detect the data loss, how to restrict/reject clients and because
the cache is partial and not complete. Once the system loads lost partitions
then ignite should allow the users to use the cache
/


I was trying to come up with a solution for the above scenario and tried to
use "TopologyValidator" & "EVT_CACHE_REBALANCE_PART_DATA_LOST" as suggested
above. But I ran into chicken and egg problem.

I was able to capture data loss event and reload data on a data loss event.
I implemented "TopologyValidator" to check the row count and block clients
from read/writing the cache, but with this approach the cache loader is also
NOT able to read/write the cache. How to allow cache loader to write to
cache, but blocks other users from using the cache.

Wonder if anyone has run into this situation and how you solved.

Or is there any cleaner way to solve this issue.

Appreciate your inputs.

Raja



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-loss-validation-TopologyValidator-tp15147p15833.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data loss validation. TopologyValidator

[vkulichenko <va...@gmail.com>]

Not sure why injection doesn't work, but you can always get Ignite instance
via Ignition.ignite().

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-loss-validation-TopologyValidator-tp15147p15214.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data loss validation. TopologyValidator

[Raja <ye...@gmail.com>]

Thank you very much for the response.

While implementing Topology Validator, I ran into the following issue. I
wasn't able to get/inject ignite instance.
Is there a way to inject/get hold of 'ignite' resource/instance within the
Topology Validator?

Thank you
Raja




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-loss-validation-TopologyValidator-tp15147p15211.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Data loss validation. TopologyValidator

[ezhuravlev <e....@gmail.com>]

Hi,

Yes, TopologyValidator was implemented exactly for these things.

Also, it's possible to listen for EVT_CACHE_REBALANCE_PART_DATA_LOST events
and implement some logic on it, but I think that TopologyValidator is better
solution.

Evgenii



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Data-loss-validation-TopologyValidator-tp15147p15172.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.