You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by syed zaheer Basha <za...@gmail.com> on 2019/07/29 11:48:21 UTC
Ignite backup/restore Cache-wise
Hi,
I have gone through the following link :
http://apache-ignite-users.70518.x6.nabble.com/Snapshot-backup-of-Ignite-native-persistance-data-tp20103.html
And came up with an idea of cache-wise backup/restore. I am describing the
steps followed :
1. Let us say an Ignite cluster exists with persistence (Native
persistence) and wal enabled on Kubernetes. I create a cache *a1* and put
*key=a* , *val=oldA* into it.
2. Now I deactivated the cluster, and copied *cache-a1* folder from storage
path of each node in Ignite cluster (keeping track of consistent node ids)
to some disk or s3 bucket and activated the cluster. This step I call as
taking a backup.
3. Now I change value of *key=a* to *val=newA* . If I fetch the *key=a* on
*cache-a1* I will now get *val=newA* .
4. Now I deactivate the cluster again, copy the previously stored
*cache-a1* folder
into the Ignite nodes (making sure I copy correct folder into correct
consistent id node folder) and activate it. This is restore step.
If I fetch *key=a* on *cache-a1* I was hoping to get *val=oldA* . But I get
*val=newA* most times. In some cases I get *val=oldA*.
What is happenning here ?
Is *WAL* bringing cache to latest state (since I am not doing anthing with
wal folder in backup/restore ) ?
Did anyone try cache-wise backup and restore without Gridgain utility ?
Does *gridgain* snapshot utility support cache-wise backup and restore ?
Re: Ignite backup/restore Cache-wise
Posted by Stanislav Lukyanov <st...@gmail.com>.
GridGain Snapshots allow you to take a backup on a live, working cluster.
If you can allow to stop the cluster activity while snapshot is being taken
you can:
- Deactive the cluster (e.g. control.sh --deactivate)
- Copy the persistence files (you would need work/binary_meta,
work/marshaller, work/db)
- Activate the cluster
To restore the data:
- Deactivate the cluster
- Place the backup files to the original location
- Activate the cluster
You may try to exclude certain caches from work/db, or only copy certain
caches, but be aware that this is not a supported use case so you may need
to figure our the correct action sequence that would work for you.
Stan
Re: Ignite backup/restore Cache-wise
Posted by syed zaheer Basha <za...@gmail.com>.
Thank you Stephen,
I want to achieve specific cache backup/restore not all caches. So If I
take backup of WAL folder as well, when I restore, all other caches state
will also get restored.
So Is there a way to distinguish WAL records of a specific cache and take
backup ?
What is the best solution to achieve specific cache backup/restore without
using Gridgain snapshot utility ?
On Mon, 29 Jul 2019 at 18:39, Stephen Darlington <
stephen.darlington@gridgain.com> wrote:
>
> Is *WAL* bringing cache to latest state (since I am not doing anthing
> with wal folder in backup/restore ) ?
>
>
> Basically, yes. The WAL contains all the changes since the last snapshot.
> To do a backup you’ll need both the data files and the WAL files. The WAL
> archives files wouldn’t hurt, too.
>
> A description of how the persistent store works can be found here:
>
>
> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
>
> GridGain supports snapshot backup/restore of all caches on a live cluster.
>
> Regards,
> Stephen
>
> On 29 Jul 2019, at 12:48, syed zaheer Basha <za...@gmail.com>
> wrote:
>
> Hi,
>
> I have gone through the following link :
>
>
> http://apache-ignite-users.70518.x6.nabble.com/Snapshot-backup-of-Ignite-native-persistance-data-tp20103.html
>
>
> And came up with an idea of cache-wise backup/restore. I am describing the
> steps followed :
>
> 1. Let us say an Ignite cluster exists with persistence (Native
> persistence) and wal enabled on Kubernetes. I create a cache *a1* and put
> *key=a* , *val=oldA* into it.
>
> 2. Now I deactivated the cluster, and copied *cache-a1* folder from
> storage path of each node in Ignite cluster (keeping track of consistent
> node ids) to some disk or s3 bucket and activated the cluster. This step I
> call as taking a backup.
>
> 3. Now I change value of *key=a* to *val=newA* . If I fetch the *key=a*
> on *cache-a1* I will now get *val=newA* .
>
> 4. Now I deactivate the cluster again, copy the previously stored
> *cache-a1* folder into the Ignite nodes (making sure I copy correct
> folder into correct consistent id node folder) and activate it. This is
> restore step.
>
> If I fetch *key=a* on *cache-a1* I was hoping to get *val=oldA* . But I
> get *val=newA* most times. In some cases I get *val=oldA*.
>
> What is happenning here ?
>
> Is *WAL* bringing cache to latest state (since I am not doing anthing
> with wal folder in backup/restore ) ?
>
> Did anyone try cache-wise backup and restore without Gridgain utility ?
>
> Does *gridgain* snapshot utility support cache-wise backup and restore ?
>
>
>
>
Re: Ignite backup/restore Cache-wise
Posted by Stephen Darlington <st...@gridgain.com>.
> Is WAL bringing cache to latest state (since I am not doing anthing with wal folder in backup/restore ) ?
Basically, yes. The WAL contains all the changes since the last snapshot. To do a backup you’ll need both the data files and the WAL files. The WAL archives files wouldn’t hurt, too.
A description of how the persistent store works can be found here:
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
GridGain supports snapshot backup/restore of all caches on a live cluster.
Regards,
Stephen
> On 29 Jul 2019, at 12:48, syed zaheer Basha <za...@gmail.com> wrote:
>
> Hi,
>
> I have gone through the following link :
>
> http://apache-ignite-users.70518.x6.nabble.com/Snapshot-backup-of-Ignite-native-persistance-data-tp20103.html <http://apache-ignite-users.70518.x6.nabble.com/Snapshot-backup-of-Ignite-native-persistance-data-tp20103.html>
>
> And came up with an idea of cache-wise backup/restore. I am describing the steps followed :
>
> 1. Let us say an Ignite cluster exists with persistence (Native persistence) and wal enabled on Kubernetes. I create a cache a1 and put key=a , val=oldA into it.
>
> 2. Now I deactivated the cluster, and copied cache-a1 folder from storage path of each node in Ignite cluster (keeping track of consistent node ids) to some disk or s3 bucket and activated the cluster. This step I call as taking a backup.
>
> 3. Now I change value of key=a to val=newA . If I fetch the key=a on cache-a1 I will now get val=newA .
>
> 4. Now I deactivate the cluster again, copy the previously stored cache-a1 folder into the Ignite nodes (making sure I copy correct folder into correct consistent id node folder) and activate it. This is restore step.
>
> If I fetch key=a on cache-a1 I was hoping to get val=oldA . But I get val=newA most times. In some cases I get val=oldA.
>
> What is happenning here ?
>
> Is WAL bringing cache to latest state (since I am not doing anthing with wal folder in backup/restore ) ?
>
> Did anyone try cache-wise backup and restore without Gridgain utility ?
>
> Does gridgain snapshot utility support cache-wise backup and restore ?