You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by "Lo, Marcus " <ma...@citi.com> on 2021/09/28 08:03:30 UTC
WAL archive compaction keeps both the unzipped and zipped wal archive
files
Hi all,
I am trying wal archive compaction by enabling walCompactionEnabled. However after enabling it I can see both the unzipped and zipped wal archive files are kept. Is this the expected behaviour? I expect that the unzipped file would be removed after zip completes. Thanks.
total 12687572
drwxrwxr-x. 2 prismuat prismuat 4096 Sep 28 15:29 .
drwxrwxr-x. 3 prismuat prismuat 26 Jun 22 11:15 ..
-rw-r--r--. 1 prismuat prismuat 2146432100 Sep 28 15:18 0000000000016561.wal
-rw-r--r--. 1 prismuat prismuat 361230 Sep 28 15:18 0000000000016561.wal.zip
-rw-r--r--. 1 prismuat prismuat 2146433779 Sep 28 15:21 0000000000016562.wal
-rw-r--r--. 1 prismuat prismuat 156 Sep 28 15:21 0000000000016562.wal.zip
-rw-r--r--. 1 prismuat prismuat 2146434466 Sep 28 15:23 0000000000016563.wal
-rw-r--r--. 1 prismuat prismuat 156 Sep 28 15:23 0000000000016563.wal.zip
-rw-r--r--. 1 prismuat prismuat 2146433326 Sep 28 15:25 0000000000016564.wal
-rw-r--r--. 1 prismuat prismuat 155 Sep 28 15:25 0000000000016564.wal.zip
-rw-r--r--. 1 prismuat prismuat 2146434984 Sep 28 15:27 0000000000016565.wal
-rw-r--r--. 1 prismuat prismuat 801519 Sep 28 15:27 0000000000016565.wal.zip
-rw-r--r--. 1 prismuat prismuat 2146434654 Sep 28 15:28 0000000000016566.wal
-rw-r--r--. 1 prismuat prismuat 112277810 Sep 28 15:29 0000000000016566.wal.zip
FYI - I am using Ignite 2.10.0
Regards,
Marcus
Re: WAL archive compaction keeps both the unzipped and zipped wal archive files
Posted by ткаленко кирилл <tk...@yandex.ru>.
I think this ticket should generally fix this problem, but it can still be present if large transactions are used.
https://issues.apache.org/jira/browse/IGNITE-14952
28.09.2021, 12:28, "Lo, Marcus" <ma...@citi.com>:
> Hi,
>
> Ah okay. I thought that wal archive compaction should reduce the disk space occupied by WAL archive, but in that case does it mean that by enabling the feature it occupies more space instead of less? Thanks.
>
> Regards,
> Marcus
>
> -----Original Message-----
> From: [External] ткаленко кирилл <tk...@yandex.ru>
> Sent: Tuesday, September 28, 2021 5:25 PM
> To: user@ignite.apache.org
> Subject: Re: WAL archive compaction keeps both the unzipped and zipped wal archive files
>
> This Message is From an External Sender This message came from outside of your organization.
>
> Yes, this is the expected behavior, these segments can still be used, for example, during a historical rebalance.
>
> 28.09.2021, 11:03, "Lo, Marcus" <ma...@citi.com>:
>> Hi all,
>>
>> I am trying wal archive compaction by enabling walCompactionEnabled. However after enabling it I can see both the unzipped and zipped wal archive files are kept. Is this the expected behaviour? I expect that the unzipped file would be removed after zip completes. Thanks.
>>
>> total 12687572
>>
>> drwxrwxr-x. 2 prismuat prismuat 4096 Sep 28 15:29 .
>>
>> drwxrwxr-x. 3 prismuat prismuat 26 Jun 22 11:15 ..
>>
>> -rw-r--r--. 1 prismuat prismuat 2146432100 Sep 28 15:18 0000000000016561.wal
>>
>> -rw-r--r--. 1 prismuat prismuat 361230 Sep 28 15:18 https://urldefense.com/v3/__http://0000000000016561.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFL8-rv6RQ$
>>
>> -rw-r--r--. 1 prismuat prismuat 2146433779 Sep 28 15:21 0000000000016562.wal
>>
>> -rw-r--r--. 1 prismuat prismuat 156 Sep 28 15:21 https://urldefense.com/v3/__http://0000000000016562.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFJnjyDdZw$
>>
>> -rw-r--r--. 1 prismuat prismuat 2146434466 Sep 28 15:23 0000000000016563.wal
>>
>> -rw-r--r--. 1 prismuat prismuat 156 Sep 28 15:23 https://urldefense.com/v3/__http://0000000000016563.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFITotdKOg$
>>
>> -rw-r--r--. 1 prismuat prismuat 2146433326 Sep 28 15:25 0000000000016564.wal
>>
>> -rw-r--r--. 1 prismuat prismuat 155 Sep 28 15:25 https://urldefense.com/v3/__http://0000000000016564.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFJ784kQhA$
>>
>> -rw-r--r--. 1 prismuat prismuat 2146434984 Sep 28 15:27 0000000000016565.wal
>>
>> -rw-r--r--. 1 prismuat prismuat 801519 Sep 28 15:27 https://urldefense.com/v3/__http://0000000000016565.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFKo57WkXw$
>>
>> -rw-r--r--. 1 prismuat prismuat 2146434654 Sep 28 15:28 0000000000016566.wal
>>
>> -rw-r--r--. 1 prismuat prismuat 112277810 Sep 28 15:29 https://urldefense.com/v3/__http://0000000000016566.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFKukYneNw$
>>
>> FYI – I am using Ignite 2.10.0
>>
>> Regards,
>>
>> Marcus
RE: WAL archive compaction keeps both the unzipped and zipped wal
archive files
Posted by "Lo, Marcus " <ma...@citi.com>.
Hi,
Ah okay. I thought that wal archive compaction should reduce the disk space occupied by WAL archive, but in that case does it mean that by enabling the feature it occupies more space instead of less? Thanks.
Regards,
Marcus
-----Original Message-----
From: [External] ткаленко кирилл <tk...@yandex.ru>
Sent: Tuesday, September 28, 2021 5:25 PM
To: user@ignite.apache.org
Subject: Re: WAL archive compaction keeps both the unzipped and zipped wal archive files
This Message is From an External Sender This message came from outside of your organization.
Yes, this is the expected behavior, these segments can still be used, for example, during a historical rebalance.
28.09.2021, 11:03, "Lo, Marcus" <ma...@citi.com>:
> Hi all,
>
> I am trying wal archive compaction by enabling walCompactionEnabled. However after enabling it I can see both the unzipped and zipped wal archive files are kept. Is this the expected behaviour? I expect that the unzipped file would be removed after zip completes. Thanks.
>
> total 12687572
>
> drwxrwxr-x. 2 prismuat prismuat 4096 Sep 28 15:29 .
>
> drwxrwxr-x. 3 prismuat prismuat 26 Jun 22 11:15 ..
>
> -rw-r--r--. 1 prismuat prismuat 2146432100 Sep 28 15:18 0000000000016561.wal
>
> -rw-r--r--. 1 prismuat prismuat 361230 Sep 28 15:18 https://urldefense.com/v3/__http://0000000000016561.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFL8-rv6RQ$
>
> -rw-r--r--. 1 prismuat prismuat 2146433779 Sep 28 15:21 0000000000016562.wal
>
> -rw-r--r--. 1 prismuat prismuat 156 Sep 28 15:21 https://urldefense.com/v3/__http://0000000000016562.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFJnjyDdZw$
>
> -rw-r--r--. 1 prismuat prismuat 2146434466 Sep 28 15:23 0000000000016563.wal
>
> -rw-r--r--. 1 prismuat prismuat 156 Sep 28 15:23 https://urldefense.com/v3/__http://0000000000016563.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFITotdKOg$
>
> -rw-r--r--. 1 prismuat prismuat 2146433326 Sep 28 15:25 0000000000016564.wal
>
> -rw-r--r--. 1 prismuat prismuat 155 Sep 28 15:25 https://urldefense.com/v3/__http://0000000000016564.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFJ784kQhA$
>
> -rw-r--r--. 1 prismuat prismuat 2146434984 Sep 28 15:27 0000000000016565.wal
>
> -rw-r--r--. 1 prismuat prismuat 801519 Sep 28 15:27 https://urldefense.com/v3/__http://0000000000016565.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFKo57WkXw$
>
> -rw-r--r--. 1 prismuat prismuat 2146434654 Sep 28 15:28 0000000000016566.wal
>
> -rw-r--r--. 1 prismuat prismuat 112277810 Sep 28 15:29 https://urldefense.com/v3/__http://0000000000016566.wal.zip__;!!Jkho33Y!yzvk0eiZgK4rum-mJvtVy5FknFFUbfTTpOFbNqme-Or76Z7D26iqnFKukYneNw$
>
> FYI – I am using Ignite 2.10.0
>
> Regards,
>
> Marcus
Re: WAL archive compaction keeps both the unzipped and zipped wal archive files
Posted by ткаленко кирилл <tk...@yandex.ru>.
Yes, this is the expected behavior, these segments can still be used, for example, during a historical rebalance.
28.09.2021, 11:03, "Lo, Marcus" <ma...@citi.com>:
> Hi all,
>
> I am trying wal archive compaction by enabling walCompactionEnabled. However after enabling it I can see both the unzipped and zipped wal archive files are kept. Is this the expected behaviour? I expect that the unzipped file would be removed after zip completes. Thanks.
>
> total 12687572
>
> drwxrwxr-x. 2 prismuat prismuat 4096 Sep 28 15:29 .
>
> drwxrwxr-x. 3 prismuat prismuat 26 Jun 22 11:15 ..
>
> -rw-r--r--. 1 prismuat prismuat 2146432100 Sep 28 15:18 0000000000016561.wal
>
> -rw-r--r--. 1 prismuat prismuat 361230 Sep 28 15:18 0000000000016561.wal.zip
>
> -rw-r--r--. 1 prismuat prismuat 2146433779 Sep 28 15:21 0000000000016562.wal
>
> -rw-r--r--. 1 prismuat prismuat 156 Sep 28 15:21 0000000000016562.wal.zip
>
> -rw-r--r--. 1 prismuat prismuat 2146434466 Sep 28 15:23 0000000000016563.wal
>
> -rw-r--r--. 1 prismuat prismuat 156 Sep 28 15:23 0000000000016563.wal.zip
>
> -rw-r--r--. 1 prismuat prismuat 2146433326 Sep 28 15:25 0000000000016564.wal
>
> -rw-r--r--. 1 prismuat prismuat 155 Sep 28 15:25 0000000000016564.wal.zip
>
> -rw-r--r--. 1 prismuat prismuat 2146434984 Sep 28 15:27 0000000000016565.wal
>
> -rw-r--r--. 1 prismuat prismuat 801519 Sep 28 15:27 0000000000016565.wal.zip
>
> -rw-r--r--. 1 prismuat prismuat 2146434654 Sep 28 15:28 0000000000016566.wal
>
> -rw-r--r--. 1 prismuat prismuat 112277810 Sep 28 15:29 0000000000016566.wal.zip
>
> FYI – I am using Ignite 2.10.0
>
> Regards,
>
> Marcus