Best practices for WalArchive when using EFS

[Raymond Wilson <ra...@trimble.com>]

We are deploying an Ignite application to a Kubernetes cluster in AWS EKS
and are planning to use EFS for persistent data.

I have been reading through the tuning options for the Write Ahead Log
Archive, trying to understand if we can safely turn off the archive to save
the IO related to copying a WAL segment to the archive when it is full.

As far as I can tell, the only functional role of the archived segments
compared to the segments present in the write ahead log is to save
potential file system slowness in recreating new segments compared to
overwriting an existing segment once it has been copied to the archive.
Apart from this, a WAL with 10 segments seems to provide the same
consistency guarantees in the event of failure as a WAL with one active
segment and a WAL archive with 10 archived segments.

As we are using EFS we do not have issues with disk space so can define an
arbitrarily large WAL without the need to copy these files to a separate
archive which will contend the EFS bandwidth even if copies asynchronously
within Ignite.

Is my understanding correct here with respect to the EFS use case?

Thanks,
Raymond.




-- 
<http://www.trimble.com/>
Raymond Wilson
Solution Architect, Civil Construction Software Systems (CCSS)
11 Birmingham Drive | Christchurch, New Zealand

Re: Best practices for WalArchive when using EFS

[Denis Magda <dm...@apache.org>]

Raymond,

I agree with you that the existence of the WAL should be more than enough
to ensure data consistency.

The documentation suggests that the archives help to speed up the
rebalancing procedures (refer to the Pros and Cons of WAL Archiving
callout): https://apacheignite.readme.io/docs/write-ahead-log#wal-archive

As for this statement, "The WAL archive is used to keep WAL segments that
may be needed to recover the node after a crash", I'm not sure if it's
relevant. If Ignite supported the point-in-time-recovery feature then the
purpose of the archives would be clear - we would be able to jump back in
time to any successfully committed operation and roll back the cluster to
that moment in time. @Alexey Goncharuk, could you please clarify this
moment? It sounds like we need to revisit our documentation.

-
Denis


On Wed, Jun 17, 2020 at 4:27 AM Raymond Wilson <ra...@trimble.com>
wrote:

> We are deploying an Ignite application to a Kubernetes cluster in AWS EKS
> and are planning to use EFS for persistent data.
>
> I have been reading through the tuning options for the Write Ahead Log
> Archive, trying to understand if we can safely turn off the archive to save
> the IO related to copying a WAL segment to the archive when it is full.
>
> As far as I can tell, the only functional role of the archived segments
> compared to the segments present in the write ahead log is to save
> potential file system slowness in recreating new segments compared to
> overwriting an existing segment once it has been copied to the archive.
> Apart from this, a WAL with 10 segments seems to provide the same
> consistency guarantees in the event of failure as a WAL with one active
> segment and a WAL archive with 10 archived segments.
>
> As we are using EFS we do not have issues with disk space so can define an
> arbitrarily large WAL without the need to copy these files to a separate
> archive which will contend the EFS bandwidth even if copies asynchronously
> within Ignite.
>
> Is my understanding correct here with respect to the EFS use case?
>
> Thanks,
> Raymond.
>
>
>
>
> --
> <http://www.trimble.com/>
> Raymond Wilson
> Solution Architect, Civil Construction Software Systems (CCSS)
> 11 Birmingham Drive | Christchurch, New Zealand
>
>