Re: NATIVE PERSISTENCE: Cache data is destroyed after disable WAL and restart

[Denis Magda <dm...@apache.org>]

Hello Manu,

This is expected to avoid data inconsistency if WAS is disabled all the
times. My suggestion is as follows:

   - Use LOG_ONLY mode for the best performance and ensure you have at
   least 1 backup copy set for each cache/table to ensure consistency if a
   node gets down:
   https://apacheignite.readme.io/docs/write-ahead-log#section-wal-modes
   - Deactivate the WAL for the loading time and turn it back once the data
   is in the cluster. This speeds up performance:
   https://apacheignite.readme.io/docs/write-ahead-log#section-wal-activation-and-deactivation
   - Tune or disable WAL archiving for even better disk utilization and
   winning more on performance:
   https://apacheignite.readme.io/docs/write-ahead-log#section-tuning-wal-archive


-
Denis


On Sun, Apr 28, 2019 at 9:37 AM Manu <ma...@hotmail.com> wrote:

> Hi!
>
> I have a question, is it normal that if WAL is deactivated for a persisted
> cache when the server node(s) is restarted, the persisted content of the
> cache is completely destroyed?
>
> I need to disable WAL for large heavy ingestion processes, but eventually
> ingestion may fail (OS, machine crash), so WAL state is not re-enabled. On
> this situation if I restart a server node, cache’s persistent
> directory is deleted and recreated again, so data is lost.
>
> Thanks!
>
> This is the method that does this hell thing
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.beforeCacheGroupStart
>
>
> Process to replicate it:
>
> 1. Start one or more server nodes with native persistence enabled
> 2. Create a cache (natively persisted) and store some data
> 3. Disable WAL for cache - ignite().cluster().disableWal("TheCacheName")
> 4. Restart server/s nodes
> 5. Check cache directory was deleted and recreated again, all data was
> lost.
>
> Call stack on server node start:
> *org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.beforeCacheGroupStart*
>
> org.apache.ignite.internal.processors.cache.ClusterCachesInfo.registerCacheGroup
>
> org.apache.ignite.internal.processors.cache.ClusterCachesInfo.registerNewCache
>
> org.apache.ignite.internal.processors.cache.ClusterCachesInfo.processJoiningNode
>
> org.apache.ignite.internal.processors.cache.ClusterCachesInfo.onStart
> *org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnStart*
>
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReadyForRead
>
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead
>
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore
>
>
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead
> org.apache.ignite.internal.IgniteKernal.start
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start
> org.apache.ignite.internal.IgnitionEx.start0
> org.apache.ignite.internal.IgnitionEx.startConfigurations
> org.apache.ignite.internal.IgnitionEx.start
> org.apache.ignite.internal.IgnitionEx.start
> org.apache.ignite.internal.IgnitionEx.start
> org.apache.ignite.internal.IgnitionEx.start
> org.apache.ignite.Ignition.start
> org.apache.ignite.startup.cmdline.CommandLineStartup.main
>
> Ignite version 2.7.0
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: NATIVE PERSISTENCE: Cache data is destroyed after disable WAL and restart

[Manu <ma...@hotmail.com>]

Thanks Denis!

Regards.

Manu.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/