You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by limabean <dr...@gmail.com> on 2017/07/11 17:46:46 UTC
How does CacheStore persistence actually work?
Hi,
I saw this remark on the mailing list from vkulichenko:
>> Generally CacheStore is designed to be a single store shared between all
>> nodes.
I want to develop my own CacheStore implementation to a data store.
The data store will have a "contact point" or client running on each Ignite
node (think a Cassandra node per Ignite node as an example). I’d like each
Ignite node to “think” that its cache persistence operations are local.
If each ignite node instantiates a copy of my custom CacheStore class on
each node, and each CacheStore instance is configured to talk locally, then
it seems what I describe is possible, but I have many questions.
Assuming the above is possible:
Each Ignite Node will need the CacheStore classes in its class path. How do
my custom CacheStore classes get to each node? How and when are the custom
CacheStore classes distributed to Ignite nodes? (best practice?)
It does not seem like the cache type (local, replicated or partitioned)
would impact each node writing locally.
For partitioned caches, that would go through the primary, and for the other
cache types...they are always "primary" ?
None of the examples (in the Ignite code base or documentation) show how to
disconnect an Ignite node from a persistent store in the CacheStore examples
that I have found. How does this work ? (Not sure how to close the
connection to a database when a CacheStore connection is closed or when an
Ignite BEFORE_NODE_STOP lifecycle event occurs?)
Thank you,
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-does-CacheStore-persistence-actually-work-tp14641.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: How does CacheStore persistence actually work?
Posted by vkulichenko <va...@gmail.com>.
1. I'm not sure this is correct. It's actually assumed that CacheStore is
shared across nodes, i.e. it's a single entry point for persisting data.
Underlying store itself can be distributed or not, but this must be
transparent to Ignite. If each node writes to its own local storage, it all
gets much more complicated and it will be very easy to get inconsistent data
during topology changes, in case of rebalancing, etc. However, Ignite 2.1
will have persistence store that is going to satisfy all this requirements.
It will use local disks on all nodes, and will handle all consistency
issues. But this will be an internal implementation of course, you will not
be able to plug in your own custom implementation to achieve the same.
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-does-CacheStore-persistence-actually-work-tp14641p15016.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: How does CacheStore persistence actually work?
Posted by limabean <dr...@gmail.com>.
Thank you, Andrew. I appreciate the insights and will take them into
consideration when writing my persistence layer.
On Fri, Jul 14, 2017 at 5:15 PM, Andrew Mashenkov [via Apache Ignite Users]
<ml...@n6.nabble.com> wrote:
> Hi,
>
> 1. Ignite doesn't cate about whether persistence operations are local or
> not. It fully relies on CacheStore implementation.
> 2. CacheStore classes should be in classpath on all nodes including client
> nodes. These classes are not delivered automatically.
> 3. Writing to store always happened on primary node for atomic caches,
> and on transaction initiator node (it can be a client node) for
> transactional caches.
> 4. BEFORE_NODE_STOP occurs right before node stop process begin. You can
> try to use EVT_CACHE_STOPPED event for closing connections to db, which
> occurs when cache has stopped.
>
>
> On Tue, Jul 11, 2017 at 8:46 PM, limabean <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=14939&i=0>> wrote:
>
>> Hi,
>>
>> I saw this remark on the mailing list from vkulichenko:
>> >> Generally CacheStore is designed to be a single store shared between
>> all
>> >> nodes.
>>
>> I want to develop my own CacheStore implementation to a data store.
>> The data store will have a "contact point" or client running on each
>> Ignite
>> node (think a Cassandra node per Ignite node as an example). I’d like
>> each
>> Ignite node to “think” that its cache persistence operations are local.
>>
>> If each ignite node instantiates a copy of my custom CacheStore class on
>> each node, and each CacheStore instance is configured to talk locally,
>> then
>> it seems what I describe is possible, but I have many questions.
>>
>> Assuming the above is possible:
>> Each Ignite Node will need the CacheStore classes in its class path. How
>> do
>> my custom CacheStore classes get to each node? How and when are the
>> custom
>> CacheStore classes distributed to Ignite nodes? (best practice?)
>>
>> It does not seem like the cache type (local, replicated or partitioned)
>> would impact each node writing locally.
>> For partitioned caches, that would go through the primary, and for the
>> other
>> cache types...they are always "primary" ?
>>
>> None of the examples (in the Ignite code base or documentation) show how
>> to
>> disconnect an Ignite node from a persistent store in the CacheStore
>> examples
>> that I have found. How does this work ? (Not sure how to close the
>> connection to a database when a CacheStore connection is closed or when an
>> Ignite BEFORE_NODE_STOP lifecycle event occurs?)
>>
>> Thank you,
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/How-does-CacheStore-persistence-actually-
>> work-tp14641.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
> Regards, Andrew.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/How-does-
> CacheStore-persistence-actually-work-tp14641p14939.html
> To unsubscribe from How does CacheStore persistence actually work?, click
> here
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=14641&code=ZHJvYmluMTQzN0BnbWFpbC5jb218MTQ2NDF8MjA5NjgyMDkyNg==>
> .
> NAML
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-does-CacheStore-persistence-actually-work-tp14641p14968.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: How does CacheStore persistence actually work?
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi,
1. Ignite doesn't cate about whether persistence operations are local or
not. It fully relies on CacheStore implementation.
2. CacheStore classes should be in classpath on all nodes including client
nodes. These classes are not delivered automatically.
3. Writing to store always happened on primary node for atomic caches,
and on transaction initiator node (it can be a client node) for
transactional caches.
4. BEFORE_NODE_STOP occurs right before node stop process begin. You can
try to use EVT_CACHE_STOPPED event for closing connections to db, which
occurs when cache has stopped.
On Tue, Jul 11, 2017 at 8:46 PM, limabean <dr...@gmail.com> wrote:
> Hi,
>
> I saw this remark on the mailing list from vkulichenko:
> >> Generally CacheStore is designed to be a single store shared between all
> >> nodes.
>
> I want to develop my own CacheStore implementation to a data store.
> The data store will have a "contact point" or client running on each Ignite
> node (think a Cassandra node per Ignite node as an example). I’d like each
> Ignite node to “think” that its cache persistence operations are local.
>
> If each ignite node instantiates a copy of my custom CacheStore class on
> each node, and each CacheStore instance is configured to talk locally, then
> it seems what I describe is possible, but I have many questions.
>
> Assuming the above is possible:
> Each Ignite Node will need the CacheStore classes in its class path. How
> do
> my custom CacheStore classes get to each node? How and when are the custom
> CacheStore classes distributed to Ignite nodes? (best practice?)
>
> It does not seem like the cache type (local, replicated or partitioned)
> would impact each node writing locally.
> For partitioned caches, that would go through the primary, and for the
> other
> cache types...they are always "primary" ?
>
> None of the examples (in the Ignite code base or documentation) show how to
> disconnect an Ignite node from a persistent store in the CacheStore
> examples
> that I have found. How does this work ? (Not sure how to close the
> connection to a database when a CacheStore connection is closed or when an
> Ignite BEFORE_NODE_STOP lifecycle event occurs?)
>
> Thank you,
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/How-does-CacheStore-persistence-
> actually-work-tp14641.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
--
Best regards,
Andrey V. Mashenkov