big wal/archive file,but node file is small

[zhouxy1123 <zh...@sina.com>]

hi ,I insert 2 T log data into 4 nodes ignite cluster through data streamer. 
I found that wal/archive directory, but storage directory is small. can
someone tell me  the reason?
 the detail like this ,in one node:
8.0K	/data/sql/marshaller
40K
/data/sql/node00-c973fbc4-5cf3-4c1c-91e5-441ee32edb39/cacheGroup-default-ds-group
168K	/data/sql/node00-c973fbc4-5cf3-4c1c-91e5-441ee32edb39/cp
112K	/data/sql/node00-c973fbc4-5cf3-4c1c-91e5-441ee32edb39/metastorage
40K
/data/sql/node00-c973fbc4-5cf3-4c1c-91e5-441ee32edb39/cache-ignite-sys-cache
9.1G	/data/sql/node00-c973fbc4-5cf3-4c1c-91e5-441ee32edb39/cache-testInstId
9.1G	/data/sql/node00-c973fbc4-5cf3-4c1c-91e5-441ee32edb39
8.0K	/data/sql/binary_meta/node00-c973fbc4-5cf3-4c1c-91e5-441ee32edb39
12K	/data/sql/binary_meta
16G	/data/sql/db/wal/archive/node00-c973fbc4-5cf3-4c1c-91e5-441ee32edb39
16G	/data/sql/db/wal/archive
641M	/data/sql/db/wal/node00-c973fbc4-5cf3-4c1c-91e5-441ee32edb39
17G	/data/sql/db/wal
17G	/data/sql/db
26G	/data/sql



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: big wal/archive file,but node file is small

[aealexsandrov <ae...@gmail.com>]

Hi,

First of all read about the WAL:

https://apacheignite.readme.io/docs/write-ahead-log
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-WALHistorySize

WAL Work maximum used size: walSegmentSize * walSegments = 640Mb (default).
It's a size of your default WAL directory. You have big size of the wall
archive because it contains all the information about the previous segments.

Looks like at the moment it can't be removed safety until next issue will
not be fixed:

https://issues.apache.org/jira/browse/IGNITE-7912 it should be available in
Ignite 2.7.

However, you can try to optimize your WAL usage:

1)Disable the wall archive logging:

https://apacheignite.readme.io/docs/write-ahead-log#section-disabling-wal-archiving

2)You can try to turn off you WAL LOG during some operations. It will be
very useful to do it during the loading of the data.

You can turn off/on WAL logging for required cache using:

1)Java code
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCluster.html#disableWal-java.lang.String-

2)SQL:

ALTER TABLE tablename NOLOGGING - turn off
ALTER TABLE tablename LOGGING - turn on

BR,
Andrei




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/