Automatically Persisted Queue

[Isaeed Mohanna <is...@gmail.com>]

Hi
I am using Ignite as a compute engine, one of my tasks will process incoming
data. Therefore I need a reliable persisted store for my new data until the
task process them, I could use ignite cache or use ignite streamer however
both these are in memory and my cluster is not kept alive always but will be
completely shutdown for upgrades once a week at least.
I know its possible to define persistence for a cache into a database
however i do not want to introduce a new database in my landscape so i am
looking for a simple approach to persist my incoming data into disk.

1. Is there a way to configure Ignite cache to be automatically stored into
disk by ignite?
2. Any persisted queue solution that you can recommend for use with ignite?
Thanks
Regards,
Isaeed Mohanna



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Automatically-Persisted-Queue-tp415.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Automatically Persisted Queue

[Dmitriy Setrakyan <ds...@apache.org>]

On Thu, May 28, 2015 at 12:03 AM, Isaeed Mohanna <is...@gmail.com> wrote:

> 1. is the cachestore always updated automatically with changes to ignite
> cache? how big is the performance penalty?
>

Generally CacheStore is synchronously updated with every cache operation,
unless you configure CacheConfiguration.setWriteBehind(true). If
write-behind is enabled, then Ignite will batch store operations and
execute them asynchronously in bulks.


> 2. My Ignite cache is partitioned across the cluster of three nodes, but
> my cache store will be in each of the nodes? will the cache store contains
> the data from all of the nodes? do I need to defined a cache store per node
> or it can run only in one of the nodes?
>

CacheStore will be called for the data stored locally on an individual
node. It will contain only the data for that node.

Re: Automatically Persisted Queue

[Isaeed Mohanna <is...@gmail.com>]

Thanks for the information,
Assume i have a in memory cache named queue, i have web calls adding items
into this queue and ignite tasks processing and deleting data from this
queue
If i implemented my own CacheStore or use JDBC Cachestore to SQLite or
something similar, several questions:
1. is the cachestore always updated automatically with changes to ignite
cache? how big is the performance penalty?
2. My Ignite cache is partitioned across the cluster of three nodes, but my
cache store will be in each of the nodes? will the cache store contains the
data from all of the nodes? do I need to defined a cache store per node or
it can run only in one of the nodes?

Regards,
Isaeed Mohanna

On Thu, May 28, 2015 at 6:27 AM, dsetrakyan <ds...@apache.org> wrote:

> Isaeed Mohanna wrote
> > I know its possible to define persistence for a cache into a database
> > however i do not want to introduce a new database in my landscape so i am
> > looking for a simple approach to persist my incoming data into disk.
> >
> > 1. Is there a way to configure Ignite cache to be automatically stored
> > into disk by ignite?
> > 2. Any persisted queue solution that you can recommend for use with
> > ignite?
>
> The automatic local disk persistence is not available out of the box in
> Ignite (to my knowledge it is available in GridGain as enterprise feature).
>
> However, you can still use CacheStore. From Ignite standpoint, CacheStore
> does not have to be a database, it could be anything, including a file
> system, to which you wish to persist the data. My advise would be to either
> provide your own CacheStore implementation, or use CacheJdbcBlobStore if
> you
> don't mind plugging in any type of local JDBC database.
>
> On startup you can use IgniteCache.loadCache() method to load the cache
> from
> database. This method will delegate to CacheStore.loadCache() for which you
> can provide your own logic to control the data loading routine.
>
>
>
>
> -----
> D.
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Automatically-Persisted-Queue-tp415p420.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Automatically Persisted Queue

[dsetrakyan <ds...@apache.org>]

Isaeed Mohanna wrote
> I know its possible to define persistence for a cache into a database
> however i do not want to introduce a new database in my landscape so i am
> looking for a simple approach to persist my incoming data into disk.
> 
> 1. Is there a way to configure Ignite cache to be automatically stored
> into disk by ignite?
> 2. Any persisted queue solution that you can recommend for use with
> ignite?

The automatic local disk persistence is not available out of the box in
Ignite (to my knowledge it is available in GridGain as enterprise feature). 

However, you can still use CacheStore. From Ignite standpoint, CacheStore
does not have to be a database, it could be anything, including a file
system, to which you wish to persist the data. My advise would be to either
provide your own CacheStore implementation, or use CacheJdbcBlobStore if you
don't mind plugging in any type of local JDBC database. 

On startup you can use IgniteCache.loadCache() method to load the cache from
database. This method will delegate to CacheStore.loadCache() for which you
can provide your own logic to control the data loading routine.




-----
D.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Automatically-Persisted-Queue-tp415p420.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.