You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by Roy Teeuwen <ro...@teeuwen.be> on 2018/03/05 13:44:37 UTC
Offline compaction - performance
Hey guys,
We have upgraded to Oak 1.6.5 (AEM 6.3) and we have noticed that the offline compaction is a lot slower than it was initially in AEM 6.1. While we had compaction of 5-15 mins for 15GB at first, it now takes half an hour. Is there any performance tweaking / parameters we can use to speed it up? Below you can find the script we use to do daily offline compaction and a log file of one of those times. Usually it takes away one GB per day. Seeing as we have the possibility to take down the system once a night, we don't have to solely rely on online compaction.
Another question I have with this, would it be ok that we remove the .bak files automatically after the compaction has ran? Or why do they stay in the folder?
Script:
```
#!/bin/bash
# AEM config
AEM_PATH="/opt/aem/author"
AEM_USER="aem-user"
CRX_QUICKSTART_PATH="${AEM_PATH}/crx-quickstart"
REPOSITORY_PATH="${CRX_QUICKSTART_PATH}/repository"
SEGMENTSTORE_PATH="${REPOSITORY_PATH}/segmentstore"
AEM_STATUS="${CRX_QUICKSTART_PATH}/bin/status"
AEM_START="${CRX_QUICKSTART_PATH}/bin/start"
AEM_STOP="${CRX_QUICKSTART_PATH}/bin/stop"
# OAK Run config
OAK_RUN_JAR="/opt/aem_dependencies/aem/oak-run-1.6.5.jar"
HEAPSIZE="-Xmx12228M"
UPDATE_LIMIT="-Dupdate.limit=10000"
COMPRESS_INTERVAL="-Dcompress-interval=100000"
COMPACTION_PROGRESS_LOG="-Dcompaction-progress-log=150000"
MEMORY_MAPPED="-Dtar.memoryMapped=true"
OAK_RUN_PARAMETERS="${HEAPSIZE} ${UPDATE_LIMIT} ${COMPRESS_INTERVAL} ${COMPACTION_PROGRESS_LOG} ${MEMORY_MAPPED}"
OAK_RUN="java ${OAK_RUN_PARAMETERS} -jar ${OAK_RUN_JAR}"
if ! [ "$(whoami)" = "${AEM_USER}" ]
then
echo "Error: Maintenance task needs to be run as AEM user (${AEM_USER})."
exit 1
fi
echo "==> Starting maintenance ($(date))"
echo "==> Make sure AEM is not running"
if ${AEM_STATUS}
then
echo "==> Stopping AEM.."
${AEM_STOP}
fi
echo "==> Cleanup unrefrenced checkpoints"
${OAK_RUN} checkpoints ${SEGMENTSTORE_PATH} || exit 1
${OAK_RUN} checkpoints ${SEGMENTSTORE_PATH} rm-unreferenced || exit 1
echo "==> Compact TAR segmentstore"
${OAK_RUN} compact ${SEGMENTSTORE_PATH} || exit 1
echo "==> Starting AEM"
${AEM_START}
echo "==> Maintenance completed ($(date))"
exit 0
```
Logs:
```
==> Stopping AEM..
05.03.2018 04:00:01.716 *INFO * [main] Setting sling.home=. (command line)
05.03.2018 04:00:01.743 *INFO * [main] Sent 'stop' to /127.0.0.1:45726: OK
Application not running
==> Cleanup unrefrenced checkpoints
Apache Jackrabbit Oak 1.6.5
Checkpoints /opt/aem/author/crx-quickstart/repository/segmentstore
- 3f22ad0f-3b78-4764-a697-c323f6d95180 created 2018-03-05 04:26:30.67 expires 2020-11-29 04:26:30.67
- 5caca09a-9d6b-426d-8166-e967ca1fbf7e created 2018-03-05 04:26:30.567 expires 2020-11-29 04:26:30.567
Found 2 checkpoints
Apache Jackrabbit Oak 1.6.5
Checkpoints /opt/aem/author/crx-quickstart/repository/segmentstore
Referenced checkpoint from /:async@async is 3f22ad0f-3b78-4764-a697-c323f6d95180
Referenced checkpoint from /:async@fulltext-async is 5caca09a-9d6b-426d-8166-e967ca1fbf7e
Removed 0 checkpoints in 44ms.
==> Compact TAR segmentstore
Apache Jackrabbit Oak 1.6.5
Compacting /opt/aem/author/crx-quickstart/repository/segmentstore
With default access mode
before
Sun Mar 04 04:48:27 CET 2018, data05195a.tar
Sun Mar 04 04:51:46 CET 2018, data05213a.tar
Sun Mar 04 04:37:16 CET 2018, data05173a.tar
Sun Mar 04 04:51:22 CET 2018, data05211a.tar
Sun Mar 04 04:43:13 CET 2018, data05182a.tar
Sun Mar 04 04:55:57 CET 2018, data05226a.tar
Mon Mar 05 03:00:57 CET 2018, data05229a.tar
Sun Mar 04 04:36:03 CET 2018, data05171a.tar
Sun Mar 04 04:52:42 CET 2018, data05218a.tar
Sun Mar 04 04:34:42 CET 2018, data05169a.tar
Tue Jan 16 06:25:35 CET 2018, data01538a.tar.bak
Sun Mar 04 04:34:07 CET 2018, data05168a.tar
Sun Mar 04 04:49:31 CET 2018, data05201a.tar
Sun Mar 04 04:53:31 CET 2018, data05222a.tar
Sun Mar 04 06:11:04 CET 2018, data05228a.tar
Sun Mar 04 04:46:31 CET 2018, data05188a.tar
Sun Mar 04 04:50:08 CET 2018, data05204a.tar
Sun Mar 04 04:48:36 CET 2018, data05196a.tar
Thu Dec 14 07:14:54 CET 2017, repo.lock
Sun Mar 04 04:43:54 CET 2018, data05183a.tar
Sun Mar 04 04:49:08 CET 2018, data05199a.tar
Sun Mar 04 04:49:43 CET 2018, data05202a.tar
Fri Mar 02 04:57:25 CET 2018, data05022a.tar.2.bak
Sun Mar 04 04:41:19 CET 2018, data05179a.tar
Mon Mar 05 04:26:34 CET 2018, journal.log
Fri Mar 02 04:57:24 CET 2018, data05022a.tar.bak
Sun Mar 04 04:53:55 CET 2018, data05224a.tar
Sun Mar 04 04:49:55 CET 2018, data05203a.tar
Sun Mar 04 04:53:03 CET 2018, data05220a.tar
Mon Mar 05 04:26:45 CET 2018, manifest
Sun Mar 04 04:49:20 CET 2018, data05200a.tar
Sun Mar 04 04:42:30 CET 2018, data05181a.tar
Sun Mar 04 04:45:37 CET 2018, data05186a.tar
Sun Mar 04 04:55:57 CET 2018, data05227a.tar
Sun Mar 04 04:48:59 CET 2018, data05198a.tar
Sun Mar 04 04:37:39 CET 2018, data05174a.tar
Sun Mar 04 04:52:19 CET 2018, data05216a.tar
Sun Mar 04 04:47:43 CET 2018, data05191a.tar
Sun Mar 04 04:48:47 CET 2018, data05197a.tar
Sun Mar 04 04:45:01 CET 2018, data05185a.tar
Sun Mar 04 04:38:20 CET 2018, data05175a.tar
Sun Mar 04 04:50:42 CET 2018, data05207a.tar
Sun Mar 04 04:40:31 CET 2018, data05178a.tar
Sun Mar 04 04:52:30 CET 2018, data05217a.tar
Sun Mar 04 04:46:56 CET 2018, data05189a.tar
Sun Mar 04 04:39:05 CET 2018, data05176a.tar
Sun Mar 04 04:35:12 CET 2018, data05170a.tar
Sun Mar 04 04:50:33 CET 2018, data05206a.tar
Sun Mar 04 04:47:22 CET 2018, data05190a.tar
Sun Mar 04 04:51:34 CET 2018, data05212a.tar
Sun Mar 04 04:54:25 CET 2018, data05225a.tar
Sun Mar 04 04:36:55 CET 2018, data05172a.tar
Sun Mar 04 04:50:20 CET 2018, data05205a.tar
Sun Mar 04 04:51:58 CET 2018, data05214a.tar
Mon Mar 05 04:26:41 CET 2018, data05231a.tar
Mon Mar 05 04:19:14 CET 2018, data05230a.tar
Sun Mar 04 04:53:43 CET 2018, data05223a.tar
Sun Mar 04 04:44:31 CET 2018, data05184a.tar
Sun Mar 04 04:47:53 CET 2018, data05192a.tar
Sun Mar 04 04:39:47 CET 2018, data05177a.tar
Sun Mar 04 04:48:17 CET 2018, data05194a.tar
Sun Mar 04 04:52:09 CET 2018, data05215a.tar
Sun Mar 04 04:48:05 CET 2018, data05193a.tar
Sun Mar 04 04:52:52 CET 2018, data05219a.tar
Sun Mar 04 04:53:20 CET 2018, data05221a.tar
Sun Mar 04 04:50:52 CET 2018, data05208a.tar
Sun Mar 04 04:46:07 CET 2018, data05187a.tar
Sun Mar 04 04:51:12 CET 2018, data05210a.tar
Sun Mar 04 04:56:04 CET 2018, gc.log
Sun Mar 04 04:33:34 CET 2018, data05167a.tar
Sun Mar 04 04:41:47 CET 2018, data05180a.tar
Sun Mar 04 04:51:02 CET 2018, data05209a.tar
size 17.1 GB (17094875037 bytes)
-> compacting
04:26:46.520 [main] WARN o.a.jackrabbit.oak.segment.Compactor - Deprecated argument 'compress-interval', please use 'oak.segment.compaction.cacheSize' instead.
-> cleaning up
-> writing new journal.log: bf675b66-e8e6-4bf4-a982-dc7316323e9c:859 root 1520222043417
after
Mon Mar 05 04:51:23 CET 2018, data05285a.tar
Mon Mar 05 04:36:53 CET 2018, data05244a.tar
Mon Mar 05 04:48:54 CET 2018, data05276a.tar
Mon Mar 05 04:43:10 CET 2018, data05254a.tar
Mon Mar 05 04:53:55 CET 2018, data05291a.tar
Mon Mar 05 04:48:31 CET 2018, data05274a.tar
Mon Mar 05 04:50:47 CET 2018, data05283a.tar
Mon Mar 05 04:40:27 CET 2018, data05249a.tar
Mon Mar 05 04:52:22 CET 2018, data05288a.tar
Mon Mar 05 04:49:30 CET 2018, data05278a.tar
Mon Mar 05 04:28:56 CET 2018, data05233a.tar
Mon Mar 05 04:31:18 CET 2018, data05236a.tar
Mon Mar 05 04:52:02 CET 2018, data05287a.tar
Tue Jan 16 06:25:35 CET 2018, data01538a.tar.bak
Mon Mar 05 04:47:51 CET 2018, data05272a.tar
Mon Mar 05 04:36:02 CET 2018, data05243a.tar
Mon Mar 05 04:35:14 CET 2018, data05242a.tar
Mon Mar 05 04:42:46 CET 2018, data05253a.tar
Mon Mar 05 04:44:55 CET 2018, data05260a.tar
Mon Mar 05 04:39:07 CET 2018, data05247a.tar
Mon Mar 05 04:46:50 CET 2018, data05268a.tar
Mon Mar 05 04:44:06 CET 2018, data05256a.tar
Mon Mar 05 04:45:33 CET 2018, data05263a.tar
Mon Mar 05 04:53:22 CET 2018, data05290a.tar
Thu Dec 14 07:14:54 CET 2017, repo.lock
Mon Mar 05 04:50:19 CET 2018, data05281a.tar
Mon Mar 05 04:49:13 CET 2018, data05277a.tar
Mon Mar 05 04:45:51 CET 2018, data05264a.tar
Mon Mar 05 04:33:40 CET 2018, data05240a.tar
Mon Mar 05 04:46:08 CET 2018, data05265a.tar
Mon Mar 05 04:32:55 CET 2018, data05239a.tar
Mon Mar 05 04:41:04 CET 2018, data05250a.tar
Fri Mar 02 04:57:25 CET 2018, data05022a.tar.2.bak
Mon Mar 05 04:45:18 CET 2018, data05262a.tar
Mon Mar 05 04:30:20 CET 2018, data05235a.tar
Mon Mar 05 04:44:44 CET 2018, data05259a.tar
Mon Mar 05 04:51:43 CET 2018, data05286a.tar
Mon Mar 05 04:54:03 CET 2018, journal.log
Fri Mar 02 04:57:24 CET 2018, data05022a.tar.bak
Mon Mar 05 04:32:32 CET 2018, data05238a.tar
Mon Mar 05 04:28:18 CET 2018, data05232a.tar
Mon Mar 05 04:45:07 CET 2018, data05261a.tar
Mon Mar 05 04:47:38 CET 2018, data05271a.tar
Mon Mar 05 04:49:48 CET 2018, data05279a.tar
Mon Mar 05 04:53:56 CET 2018, manifest
Mon Mar 05 04:34:30 CET 2018, data05241a.tar
Mon Mar 05 04:46:20 CET 2018, data05266a.tar
Mon Mar 05 04:43:41 CET 2018, data05255a.tar
Mon Mar 05 04:38:21 CET 2018, data05246a.tar
Mon Mar 05 04:48:14 CET 2018, data05273a.tar
Mon Mar 05 04:52:50 CET 2018, data05289a.tar
Mon Mar 05 04:42:19 CET 2018, data05252a.tar
Mon Mar 05 04:41:47 CET 2018, data05251a.tar
Mon Mar 05 04:47:07 CET 2018, data05269a.tar
Mon Mar 05 04:50:34 CET 2018, data05282a.tar
Mon Mar 05 04:47:21 CET 2018, data05270a.tar
Mon Mar 05 04:39:50 CET 2018, data05248a.tar
Mon Mar 05 04:29:40 CET 2018, data05234a.tar
Mon Mar 05 04:44:19 CET 2018, data05257a.tar
Mon Mar 05 04:44:30 CET 2018, data05258a.tar
Mon Mar 05 04:32:13 CET 2018, data05237a.tar
Mon Mar 05 04:48:43 CET 2018, data05275a.tar
Mon Mar 05 04:50:05 CET 2018, data05280a.tar
Mon Mar 05 04:37:30 CET 2018, data05245a.tar
Mon Mar 05 04:51:02 CET 2018, data05284a.tar
Mon Mar 05 04:46:33 CET 2018, data05267a.tar
Mon Mar 05 04:54:03 CET 2018, gc.log
size 16.1 GB (16081145586 bytes)
removed files [data05228a.tar, data05222a.tar, data05167a.tar, data05184a.tar, data05214a.tar, data05192a.tar, data05200a.tar, data05198a.tar, data05183a.tar, data05189a.tar, data05205a.tar, data05175a.tar, data05230a.tar, data05206a.tar, data05191a.tar, data05197a.tar, data05174a.tar, data05177a.tar, data05188a.tar, data05171a.tar, data05182a.tar, data05169a.tar, data05224a.tar, data05227a.tar, data05221a.tar, data05213a.tar, data05193a.tar, data05185a.tar, data05190a.tar, data05216a.tar, data05219a.tar, data05196a.tar, data05199a.tar, data05195a.tar, data05211a.tar, data05187a.tar, data05203a.tar, data05208a.tar, data05186a.tar, data05172a.tar, data05180a.tar, data05202a.tar, data05210a.tar, data05178a.tar, data05209a.tar, data05217a.tar, data05225a.tar, data05194a.tar, data05207a.tar, data05218a.tar, data05229a.tar, data05220a.tar, data05168a.tar, data05223a.tar, data05226a.tar, data05179a.tar, data05176a.tar, data05231a.tar, data05170a.tar, data05215a.tar, data05201a.tar, data05181a.tar, data05173a.tar, data05204a.tar, data05212a.tar]
added files [data05259a.tar, data05267a.tar, data05245a.tar, data05237a.tar, data05253a.tar, data05261a.tar, data05283a.tar, data05244a.tar, data05258a.tar, data05291a.tar, data05236a.tar, data05275a.tar, data05260a.tar, data05252a.tar, data05289a.tar, data05232a.tar, data05246a.tar, data05243a.tar, data05238a.tar, data05235a.tar, data05240a.tar, data05279a.tar, data05262a.tar, data05268a.tar, data05276a.tar, data05251a.tar, data05284a.tar, data05270a.tar, data05254a.tar, data05273a.tar, data05281a.tar, data05265a.tar, data05249a.tar, data05257a.tar, data05250a.tar, data05256a.tar, data05264a.tar, data05239a.tar, data05287a.tar, data05272a.tar, data05242a.tar, data05248a.tar, data05241a.tar, data05280a.tar, data05247a.tar, data05255a.tar, data05278a.tar, data05233a.tar, data05286a.tar, data05271a.tar, data05274a.tar, data05288a.tar, data05263a.tar, data05266a.tar, data05285a.tar, data05290a.tar, data05277a.tar, data05282a.tar, data05269a.tar, data05234a.tar]
Compaction succeeded in 27.29 min (1637s).
```
Greets,
Roy
Re: Offline compaction - performance
Posted by Michael Dürig <mi...@gmail.com>.
Correct, when an instance is shutdown in an unclean way (i.e. kill -9)
a backup of the most recent tar file will most likely be created on
the next start.
Michael
On 5 March 2018 at 17:07, Roy Teeuwen <ro...@teeuwen.be> wrote:
> Hey Michael,
>
> Good to hear because we are updating to 1.6.6. in 3 weeks! Let's hope it helps.
>
> About the .bak files, do I understand it correctly then that the .bak files are only created when something went wrong on shutdown?
>
> Greets,
> Roy
>
>> On 5 Mar 2018, at 16:59, Michael Dürig <mi...@gmail.com> wrote:
>>
>> Roy,
>>
>> One thing that might help is upgrading to 1.6.6 to get the fix for
>> OAK-6642 /OAK-6110. With that fix you should be able to run within a
>> 2GB heap and compaction times should go down.
>>
>> Re. the bak files: from an operational POV it is safe to remove them.
>> Those files are kept in the case an instances in shut down in an non
>> clean way and needs to recover the most recent tar file that hasn't
>> been properly closed. There is up to 5 seconds worth of data that
>> might get lost in that process. The bak files *might* contain parts of
>> this data and it *might* be possible to recover *some* of it manually.
>> If this is not needed (I haven't seen a case where it was so far), you
>> can safely remove them.
>>
>> Michael
>>
>> .
>>
>> On 5 March 2018 at 14:44, Roy Teeuwen <ro...@teeuwen.be> wrote:
>>> Hey guys,
>>>
>>> We have upgraded to Oak 1.6.5 (AEM 6.3) and we have noticed that the offline compaction is a lot slower than it was initially in AEM 6.1. While we had compaction of 5-15 mins for 15GB at first, it now takes half an hour. Is there any performance tweaking / parameters we can use to speed it up? Below you can find the script we use to do daily offline compaction and a log file of one of those times. Usually it takes away one GB per day. Seeing as we have the possibility to take down the system once a night, we don't have to solely rely on online compaction.
>>>
>>> Another question I have with this, would it be ok that we remove the .bak files automatically after the compaction has ran? Or why do they stay in the folder?
>>>
>>> Script:
>>> ```
>>> #!/bin/bash
>>>
>>> # AEM config
>>> AEM_PATH="/opt/aem/author"
>>> AEM_USER="aem-user"
>>> CRX_QUICKSTART_PATH="${AEM_PATH}/crx-quickstart"
>>> REPOSITORY_PATH="${CRX_QUICKSTART_PATH}/repository"
>>> SEGMENTSTORE_PATH="${REPOSITORY_PATH}/segmentstore"
>>> AEM_STATUS="${CRX_QUICKSTART_PATH}/bin/status"
>>> AEM_START="${CRX_QUICKSTART_PATH}/bin/start"
>>> AEM_STOP="${CRX_QUICKSTART_PATH}/bin/stop"
>>>
>>> # OAK Run config
>>> OAK_RUN_JAR="/opt/aem_dependencies/aem/oak-run-1.6.5.jar"
>>> HEAPSIZE="-Xmx12228M"
>>> UPDATE_LIMIT="-Dupdate.limit=10000"
>>> COMPRESS_INTERVAL="-Dcompress-interval=100000"
>>> COMPACTION_PROGRESS_LOG="-Dcompaction-progress-log=150000"
>>> MEMORY_MAPPED="-Dtar.memoryMapped=true"
>>> OAK_RUN_PARAMETERS="${HEAPSIZE} ${UPDATE_LIMIT} ${COMPRESS_INTERVAL} ${COMPACTION_PROGRESS_LOG} ${MEMORY_MAPPED}"
>>> OAK_RUN="java ${OAK_RUN_PARAMETERS} -jar ${OAK_RUN_JAR}"
>>>
>>> if ! [ "$(whoami)" = "${AEM_USER}" ]
>>> then
>>> echo "Error: Maintenance task needs to be run as AEM user (${AEM_USER})."
>>> exit 1
>>> fi
>>>
>>> echo "==> Starting maintenance ($(date))"
>>>
>>> echo "==> Make sure AEM is not running"
>>> if ${AEM_STATUS}
>>> then
>>> echo "==> Stopping AEM.."
>>> ${AEM_STOP}
>>> fi
>>>
>>> echo "==> Cleanup unrefrenced checkpoints"
>>> ${OAK_RUN} checkpoints ${SEGMENTSTORE_PATH} || exit 1
>>> ${OAK_RUN} checkpoints ${SEGMENTSTORE_PATH} rm-unreferenced || exit 1
>>>
>>> echo "==> Compact TAR segmentstore"
>>> ${OAK_RUN} compact ${SEGMENTSTORE_PATH} || exit 1
>>>
>>> echo "==> Starting AEM"
>>> ${AEM_START}
>>>
>>> echo "==> Maintenance completed ($(date))"
>>>
>>> exit 0
>>>
>>> ```
>>>
>>> Logs:
>>>
>>> ```
>>> ==> Stopping AEM..
>>> 05.03.2018 04:00:01.716 *INFO * [main] Setting sling.home=. (command line)
>>> 05.03.2018 04:00:01.743 *INFO * [main] Sent 'stop' to /127.0.0.1:45726: OK
>>> Application not running
>>> ==> Cleanup unrefrenced checkpoints
>>> Apache Jackrabbit Oak 1.6.5
>>> Checkpoints /opt/aem/author/crx-quickstart/repository/segmentstore
>>> - 3f22ad0f-3b78-4764-a697-c323f6d95180 created 2018-03-05 04:26:30.67 expires 2020-11-29 04:26:30.67
>>> - 5caca09a-9d6b-426d-8166-e967ca1fbf7e created 2018-03-05 04:26:30.567 expires 2020-11-29 04:26:30.567
>>> Found 2 checkpoints
>>> Apache Jackrabbit Oak 1.6.5
>>> Checkpoints /opt/aem/author/crx-quickstart/repository/segmentstore
>>> Referenced checkpoint from /:async@async is 3f22ad0f-3b78-4764-a697-c323f6d95180
>>> Referenced checkpoint from /:async@fulltext-async is 5caca09a-9d6b-426d-8166-e967ca1fbf7e
>>> Removed 0 checkpoints in 44ms.
>>> ==> Compact TAR segmentstore
>>> Apache Jackrabbit Oak 1.6.5
>>> Compacting /opt/aem/author/crx-quickstart/repository/segmentstore
>>> With default access mode
>>> before
>>> Sun Mar 04 04:48:27 CET 2018, data05195a.tar
>>> Sun Mar 04 04:51:46 CET 2018, data05213a.tar
>>> Sun Mar 04 04:37:16 CET 2018, data05173a.tar
>>> Sun Mar 04 04:51:22 CET 2018, data05211a.tar
>>> Sun Mar 04 04:43:13 CET 2018, data05182a.tar
>>> Sun Mar 04 04:55:57 CET 2018, data05226a.tar
>>> Mon Mar 05 03:00:57 CET 2018, data05229a.tar
>>> Sun Mar 04 04:36:03 CET 2018, data05171a.tar
>>> Sun Mar 04 04:52:42 CET 2018, data05218a.tar
>>> Sun Mar 04 04:34:42 CET 2018, data05169a.tar
>>> Tue Jan 16 06:25:35 CET 2018, data01538a.tar.bak
>>> Sun Mar 04 04:34:07 CET 2018, data05168a.tar
>>> Sun Mar 04 04:49:31 CET 2018, data05201a.tar
>>> Sun Mar 04 04:53:31 CET 2018, data05222a.tar
>>> Sun Mar 04 06:11:04 CET 2018, data05228a.tar
>>> Sun Mar 04 04:46:31 CET 2018, data05188a.tar
>>> Sun Mar 04 04:50:08 CET 2018, data05204a.tar
>>> Sun Mar 04 04:48:36 CET 2018, data05196a.tar
>>> Thu Dec 14 07:14:54 CET 2017, repo.lock
>>> Sun Mar 04 04:43:54 CET 2018, data05183a.tar
>>> Sun Mar 04 04:49:08 CET 2018, data05199a.tar
>>> Sun Mar 04 04:49:43 CET 2018, data05202a.tar
>>> Fri Mar 02 04:57:25 CET 2018, data05022a.tar.2.bak
>>> Sun Mar 04 04:41:19 CET 2018, data05179a.tar
>>> Mon Mar 05 04:26:34 CET 2018, journal.log
>>> Fri Mar 02 04:57:24 CET 2018, data05022a.tar.bak
>>> Sun Mar 04 04:53:55 CET 2018, data05224a.tar
>>> Sun Mar 04 04:49:55 CET 2018, data05203a.tar
>>> Sun Mar 04 04:53:03 CET 2018, data05220a.tar
>>> Mon Mar 05 04:26:45 CET 2018, manifest
>>> Sun Mar 04 04:49:20 CET 2018, data05200a.tar
>>> Sun Mar 04 04:42:30 CET 2018, data05181a.tar
>>> Sun Mar 04 04:45:37 CET 2018, data05186a.tar
>>> Sun Mar 04 04:55:57 CET 2018, data05227a.tar
>>> Sun Mar 04 04:48:59 CET 2018, data05198a.tar
>>> Sun Mar 04 04:37:39 CET 2018, data05174a.tar
>>> Sun Mar 04 04:52:19 CET 2018, data05216a.tar
>>> Sun Mar 04 04:47:43 CET 2018, data05191a.tar
>>> Sun Mar 04 04:48:47 CET 2018, data05197a.tar
>>> Sun Mar 04 04:45:01 CET 2018, data05185a.tar
>>> Sun Mar 04 04:38:20 CET 2018, data05175a.tar
>>> Sun Mar 04 04:50:42 CET 2018, data05207a.tar
>>> Sun Mar 04 04:40:31 CET 2018, data05178a.tar
>>> Sun Mar 04 04:52:30 CET 2018, data05217a.tar
>>> Sun Mar 04 04:46:56 CET 2018, data05189a.tar
>>> Sun Mar 04 04:39:05 CET 2018, data05176a.tar
>>> Sun Mar 04 04:35:12 CET 2018, data05170a.tar
>>> Sun Mar 04 04:50:33 CET 2018, data05206a.tar
>>> Sun Mar 04 04:47:22 CET 2018, data05190a.tar
>>> Sun Mar 04 04:51:34 CET 2018, data05212a.tar
>>> Sun Mar 04 04:54:25 CET 2018, data05225a.tar
>>> Sun Mar 04 04:36:55 CET 2018, data05172a.tar
>>> Sun Mar 04 04:50:20 CET 2018, data05205a.tar
>>> Sun Mar 04 04:51:58 CET 2018, data05214a.tar
>>> Mon Mar 05 04:26:41 CET 2018, data05231a.tar
>>> Mon Mar 05 04:19:14 CET 2018, data05230a.tar
>>> Sun Mar 04 04:53:43 CET 2018, data05223a.tar
>>> Sun Mar 04 04:44:31 CET 2018, data05184a.tar
>>> Sun Mar 04 04:47:53 CET 2018, data05192a.tar
>>> Sun Mar 04 04:39:47 CET 2018, data05177a.tar
>>> Sun Mar 04 04:48:17 CET 2018, data05194a.tar
>>> Sun Mar 04 04:52:09 CET 2018, data05215a.tar
>>> Sun Mar 04 04:48:05 CET 2018, data05193a.tar
>>> Sun Mar 04 04:52:52 CET 2018, data05219a.tar
>>> Sun Mar 04 04:53:20 CET 2018, data05221a.tar
>>> Sun Mar 04 04:50:52 CET 2018, data05208a.tar
>>> Sun Mar 04 04:46:07 CET 2018, data05187a.tar
>>> Sun Mar 04 04:51:12 CET 2018, data05210a.tar
>>> Sun Mar 04 04:56:04 CET 2018, gc.log
>>> Sun Mar 04 04:33:34 CET 2018, data05167a.tar
>>> Sun Mar 04 04:41:47 CET 2018, data05180a.tar
>>> Sun Mar 04 04:51:02 CET 2018, data05209a.tar
>>> size 17.1 GB (17094875037 bytes)
>>> -> compacting
>>> 04:26:46.520 [main] WARN o.a.jackrabbit.oak.segment.Compactor - Deprecated argument 'compress-interval', please use 'oak.segment.compaction.cacheSize' instead.
>>> -> cleaning up
>>> -> writing new journal.log: bf675b66-e8e6-4bf4-a982-dc7316323e9c:859 root 1520222043417
>>>
>>> after
>>> Mon Mar 05 04:51:23 CET 2018, data05285a.tar
>>> Mon Mar 05 04:36:53 CET 2018, data05244a.tar
>>> Mon Mar 05 04:48:54 CET 2018, data05276a.tar
>>> Mon Mar 05 04:43:10 CET 2018, data05254a.tar
>>> Mon Mar 05 04:53:55 CET 2018, data05291a.tar
>>> Mon Mar 05 04:48:31 CET 2018, data05274a.tar
>>> Mon Mar 05 04:50:47 CET 2018, data05283a.tar
>>> Mon Mar 05 04:40:27 CET 2018, data05249a.tar
>>> Mon Mar 05 04:52:22 CET 2018, data05288a.tar
>>> Mon Mar 05 04:49:30 CET 2018, data05278a.tar
>>> Mon Mar 05 04:28:56 CET 2018, data05233a.tar
>>> Mon Mar 05 04:31:18 CET 2018, data05236a.tar
>>> Mon Mar 05 04:52:02 CET 2018, data05287a.tar
>>> Tue Jan 16 06:25:35 CET 2018, data01538a.tar.bak
>>> Mon Mar 05 04:47:51 CET 2018, data05272a.tar
>>> Mon Mar 05 04:36:02 CET 2018, data05243a.tar
>>> Mon Mar 05 04:35:14 CET 2018, data05242a.tar
>>> Mon Mar 05 04:42:46 CET 2018, data05253a.tar
>>> Mon Mar 05 04:44:55 CET 2018, data05260a.tar
>>> Mon Mar 05 04:39:07 CET 2018, data05247a.tar
>>> Mon Mar 05 04:46:50 CET 2018, data05268a.tar
>>> Mon Mar 05 04:44:06 CET 2018, data05256a.tar
>>> Mon Mar 05 04:45:33 CET 2018, data05263a.tar
>>> Mon Mar 05 04:53:22 CET 2018, data05290a.tar
>>> Thu Dec 14 07:14:54 CET 2017, repo.lock
>>> Mon Mar 05 04:50:19 CET 2018, data05281a.tar
>>> Mon Mar 05 04:49:13 CET 2018, data05277a.tar
>>> Mon Mar 05 04:45:51 CET 2018, data05264a.tar
>>> Mon Mar 05 04:33:40 CET 2018, data05240a.tar
>>> Mon Mar 05 04:46:08 CET 2018, data05265a.tar
>>> Mon Mar 05 04:32:55 CET 2018, data05239a.tar
>>> Mon Mar 05 04:41:04 CET 2018, data05250a.tar
>>> Fri Mar 02 04:57:25 CET 2018, data05022a.tar.2.bak
>>> Mon Mar 05 04:45:18 CET 2018, data05262a.tar
>>> Mon Mar 05 04:30:20 CET 2018, data05235a.tar
>>> Mon Mar 05 04:44:44 CET 2018, data05259a.tar
>>> Mon Mar 05 04:51:43 CET 2018, data05286a.tar
>>> Mon Mar 05 04:54:03 CET 2018, journal.log
>>> Fri Mar 02 04:57:24 CET 2018, data05022a.tar.bak
>>> Mon Mar 05 04:32:32 CET 2018, data05238a.tar
>>> Mon Mar 05 04:28:18 CET 2018, data05232a.tar
>>> Mon Mar 05 04:45:07 CET 2018, data05261a.tar
>>> Mon Mar 05 04:47:38 CET 2018, data05271a.tar
>>> Mon Mar 05 04:49:48 CET 2018, data05279a.tar
>>> Mon Mar 05 04:53:56 CET 2018, manifest
>>> Mon Mar 05 04:34:30 CET 2018, data05241a.tar
>>> Mon Mar 05 04:46:20 CET 2018, data05266a.tar
>>> Mon Mar 05 04:43:41 CET 2018, data05255a.tar
>>> Mon Mar 05 04:38:21 CET 2018, data05246a.tar
>>> Mon Mar 05 04:48:14 CET 2018, data05273a.tar
>>> Mon Mar 05 04:52:50 CET 2018, data05289a.tar
>>> Mon Mar 05 04:42:19 CET 2018, data05252a.tar
>>> Mon Mar 05 04:41:47 CET 2018, data05251a.tar
>>> Mon Mar 05 04:47:07 CET 2018, data05269a.tar
>>> Mon Mar 05 04:50:34 CET 2018, data05282a.tar
>>> Mon Mar 05 04:47:21 CET 2018, data05270a.tar
>>> Mon Mar 05 04:39:50 CET 2018, data05248a.tar
>>> Mon Mar 05 04:29:40 CET 2018, data05234a.tar
>>> Mon Mar 05 04:44:19 CET 2018, data05257a.tar
>>> Mon Mar 05 04:44:30 CET 2018, data05258a.tar
>>> Mon Mar 05 04:32:13 CET 2018, data05237a.tar
>>> Mon Mar 05 04:48:43 CET 2018, data05275a.tar
>>> Mon Mar 05 04:50:05 CET 2018, data05280a.tar
>>> Mon Mar 05 04:37:30 CET 2018, data05245a.tar
>>> Mon Mar 05 04:51:02 CET 2018, data05284a.tar
>>> Mon Mar 05 04:46:33 CET 2018, data05267a.tar
>>> Mon Mar 05 04:54:03 CET 2018, gc.log
>>> size 16.1 GB (16081145586 bytes)
>>> removed files [data05228a.tar, data05222a.tar, data05167a.tar, data05184a.tar, data05214a.tar, data05192a.tar, data05200a.tar, data05198a.tar, data05183a.tar, data05189a.tar, data05205a.tar, data05175a.tar, data05230a.tar, data05206a.tar, data05191a.tar, data05197a.tar, data05174a.tar, data05177a.tar, data05188a.tar, data05171a.tar, data05182a.tar, data05169a.tar, data05224a.tar, data05227a.tar, data05221a.tar, data05213a.tar, data05193a.tar, data05185a.tar, data05190a.tar, data05216a.tar, data05219a.tar, data05196a.tar, data05199a.tar, data05195a.tar, data05211a.tar, data05187a.tar, data05203a.tar, data05208a.tar, data05186a.tar, data05172a.tar, data05180a.tar, data05202a.tar, data05210a.tar, data05178a.tar, data05209a.tar, data05217a.tar, data05225a.tar, data05194a.tar, data05207a.tar, data05218a.tar, data05229a.tar, data05220a.tar, data05168a.tar, data05223a.tar, data05226a.tar, data05179a.tar, data05176a.tar, data05231a.tar, data05170a.tar, data05215a.tar, data05201a.tar, data05181a.tar, data05173a.tar, data05204a.tar, data05212a.tar]
>>> added files [data05259a.tar, data05267a.tar, data05245a.tar, data05237a.tar, data05253a.tar, data05261a.tar, data05283a.tar, data05244a.tar, data05258a.tar, data05291a.tar, data05236a.tar, data05275a.tar, data05260a.tar, data05252a.tar, data05289a.tar, data05232a.tar, data05246a.tar, data05243a.tar, data05238a.tar, data05235a.tar, data05240a.tar, data05279a.tar, data05262a.tar, data05268a.tar, data05276a.tar, data05251a.tar, data05284a.tar, data05270a.tar, data05254a.tar, data05273a.tar, data05281a.tar, data05265a.tar, data05249a.tar, data05257a.tar, data05250a.tar, data05256a.tar, data05264a.tar, data05239a.tar, data05287a.tar, data05272a.tar, data05242a.tar, data05248a.tar, data05241a.tar, data05280a.tar, data05247a.tar, data05255a.tar, data05278a.tar, data05233a.tar, data05286a.tar, data05271a.tar, data05274a.tar, data05288a.tar, data05263a.tar, data05266a.tar, data05285a.tar, data05290a.tar, data05277a.tar, data05282a.tar, data05269a.tar, data05234a.tar]
>>> Compaction succeeded in 27.29 min (1637s).
>>>
>>> ```
>>>
>>> Greets,
>>> Roy
>
Re: Offline compaction - performance
Posted by Roy Teeuwen <ro...@teeuwen.be>.
Hey Michael,
Good to hear because we are updating to 1.6.6. in 3 weeks! Let's hope it helps.
About the .bak files, do I understand it correctly then that the .bak files are only created when something went wrong on shutdown?
Greets,
Roy
> On 5 Mar 2018, at 16:59, Michael Dürig <mi...@gmail.com> wrote:
>
> Roy,
>
> One thing that might help is upgrading to 1.6.6 to get the fix for
> OAK-6642 /OAK-6110. With that fix you should be able to run within a
> 2GB heap and compaction times should go down.
>
> Re. the bak files: from an operational POV it is safe to remove them.
> Those files are kept in the case an instances in shut down in an non
> clean way and needs to recover the most recent tar file that hasn't
> been properly closed. There is up to 5 seconds worth of data that
> might get lost in that process. The bak files *might* contain parts of
> this data and it *might* be possible to recover *some* of it manually.
> If this is not needed (I haven't seen a case where it was so far), you
> can safely remove them.
>
> Michael
>
> .
>
> On 5 March 2018 at 14:44, Roy Teeuwen <ro...@teeuwen.be> wrote:
>> Hey guys,
>>
>> We have upgraded to Oak 1.6.5 (AEM 6.3) and we have noticed that the offline compaction is a lot slower than it was initially in AEM 6.1. While we had compaction of 5-15 mins for 15GB at first, it now takes half an hour. Is there any performance tweaking / parameters we can use to speed it up? Below you can find the script we use to do daily offline compaction and a log file of one of those times. Usually it takes away one GB per day. Seeing as we have the possibility to take down the system once a night, we don't have to solely rely on online compaction.
>>
>> Another question I have with this, would it be ok that we remove the .bak files automatically after the compaction has ran? Or why do they stay in the folder?
>>
>> Script:
>> ```
>> #!/bin/bash
>>
>> # AEM config
>> AEM_PATH="/opt/aem/author"
>> AEM_USER="aem-user"
>> CRX_QUICKSTART_PATH="${AEM_PATH}/crx-quickstart"
>> REPOSITORY_PATH="${CRX_QUICKSTART_PATH}/repository"
>> SEGMENTSTORE_PATH="${REPOSITORY_PATH}/segmentstore"
>> AEM_STATUS="${CRX_QUICKSTART_PATH}/bin/status"
>> AEM_START="${CRX_QUICKSTART_PATH}/bin/start"
>> AEM_STOP="${CRX_QUICKSTART_PATH}/bin/stop"
>>
>> # OAK Run config
>> OAK_RUN_JAR="/opt/aem_dependencies/aem/oak-run-1.6.5.jar"
>> HEAPSIZE="-Xmx12228M"
>> UPDATE_LIMIT="-Dupdate.limit=10000"
>> COMPRESS_INTERVAL="-Dcompress-interval=100000"
>> COMPACTION_PROGRESS_LOG="-Dcompaction-progress-log=150000"
>> MEMORY_MAPPED="-Dtar.memoryMapped=true"
>> OAK_RUN_PARAMETERS="${HEAPSIZE} ${UPDATE_LIMIT} ${COMPRESS_INTERVAL} ${COMPACTION_PROGRESS_LOG} ${MEMORY_MAPPED}"
>> OAK_RUN="java ${OAK_RUN_PARAMETERS} -jar ${OAK_RUN_JAR}"
>>
>> if ! [ "$(whoami)" = "${AEM_USER}" ]
>> then
>> echo "Error: Maintenance task needs to be run as AEM user (${AEM_USER})."
>> exit 1
>> fi
>>
>> echo "==> Starting maintenance ($(date))"
>>
>> echo "==> Make sure AEM is not running"
>> if ${AEM_STATUS}
>> then
>> echo "==> Stopping AEM.."
>> ${AEM_STOP}
>> fi
>>
>> echo "==> Cleanup unrefrenced checkpoints"
>> ${OAK_RUN} checkpoints ${SEGMENTSTORE_PATH} || exit 1
>> ${OAK_RUN} checkpoints ${SEGMENTSTORE_PATH} rm-unreferenced || exit 1
>>
>> echo "==> Compact TAR segmentstore"
>> ${OAK_RUN} compact ${SEGMENTSTORE_PATH} || exit 1
>>
>> echo "==> Starting AEM"
>> ${AEM_START}
>>
>> echo "==> Maintenance completed ($(date))"
>>
>> exit 0
>>
>> ```
>>
>> Logs:
>>
>> ```
>> ==> Stopping AEM..
>> 05.03.2018 04:00:01.716 *INFO * [main] Setting sling.home=. (command line)
>> 05.03.2018 04:00:01.743 *INFO * [main] Sent 'stop' to /127.0.0.1:45726: OK
>> Application not running
>> ==> Cleanup unrefrenced checkpoints
>> Apache Jackrabbit Oak 1.6.5
>> Checkpoints /opt/aem/author/crx-quickstart/repository/segmentstore
>> - 3f22ad0f-3b78-4764-a697-c323f6d95180 created 2018-03-05 04:26:30.67 expires 2020-11-29 04:26:30.67
>> - 5caca09a-9d6b-426d-8166-e967ca1fbf7e created 2018-03-05 04:26:30.567 expires 2020-11-29 04:26:30.567
>> Found 2 checkpoints
>> Apache Jackrabbit Oak 1.6.5
>> Checkpoints /opt/aem/author/crx-quickstart/repository/segmentstore
>> Referenced checkpoint from /:async@async is 3f22ad0f-3b78-4764-a697-c323f6d95180
>> Referenced checkpoint from /:async@fulltext-async is 5caca09a-9d6b-426d-8166-e967ca1fbf7e
>> Removed 0 checkpoints in 44ms.
>> ==> Compact TAR segmentstore
>> Apache Jackrabbit Oak 1.6.5
>> Compacting /opt/aem/author/crx-quickstart/repository/segmentstore
>> With default access mode
>> before
>> Sun Mar 04 04:48:27 CET 2018, data05195a.tar
>> Sun Mar 04 04:51:46 CET 2018, data05213a.tar
>> Sun Mar 04 04:37:16 CET 2018, data05173a.tar
>> Sun Mar 04 04:51:22 CET 2018, data05211a.tar
>> Sun Mar 04 04:43:13 CET 2018, data05182a.tar
>> Sun Mar 04 04:55:57 CET 2018, data05226a.tar
>> Mon Mar 05 03:00:57 CET 2018, data05229a.tar
>> Sun Mar 04 04:36:03 CET 2018, data05171a.tar
>> Sun Mar 04 04:52:42 CET 2018, data05218a.tar
>> Sun Mar 04 04:34:42 CET 2018, data05169a.tar
>> Tue Jan 16 06:25:35 CET 2018, data01538a.tar.bak
>> Sun Mar 04 04:34:07 CET 2018, data05168a.tar
>> Sun Mar 04 04:49:31 CET 2018, data05201a.tar
>> Sun Mar 04 04:53:31 CET 2018, data05222a.tar
>> Sun Mar 04 06:11:04 CET 2018, data05228a.tar
>> Sun Mar 04 04:46:31 CET 2018, data05188a.tar
>> Sun Mar 04 04:50:08 CET 2018, data05204a.tar
>> Sun Mar 04 04:48:36 CET 2018, data05196a.tar
>> Thu Dec 14 07:14:54 CET 2017, repo.lock
>> Sun Mar 04 04:43:54 CET 2018, data05183a.tar
>> Sun Mar 04 04:49:08 CET 2018, data05199a.tar
>> Sun Mar 04 04:49:43 CET 2018, data05202a.tar
>> Fri Mar 02 04:57:25 CET 2018, data05022a.tar.2.bak
>> Sun Mar 04 04:41:19 CET 2018, data05179a.tar
>> Mon Mar 05 04:26:34 CET 2018, journal.log
>> Fri Mar 02 04:57:24 CET 2018, data05022a.tar.bak
>> Sun Mar 04 04:53:55 CET 2018, data05224a.tar
>> Sun Mar 04 04:49:55 CET 2018, data05203a.tar
>> Sun Mar 04 04:53:03 CET 2018, data05220a.tar
>> Mon Mar 05 04:26:45 CET 2018, manifest
>> Sun Mar 04 04:49:20 CET 2018, data05200a.tar
>> Sun Mar 04 04:42:30 CET 2018, data05181a.tar
>> Sun Mar 04 04:45:37 CET 2018, data05186a.tar
>> Sun Mar 04 04:55:57 CET 2018, data05227a.tar
>> Sun Mar 04 04:48:59 CET 2018, data05198a.tar
>> Sun Mar 04 04:37:39 CET 2018, data05174a.tar
>> Sun Mar 04 04:52:19 CET 2018, data05216a.tar
>> Sun Mar 04 04:47:43 CET 2018, data05191a.tar
>> Sun Mar 04 04:48:47 CET 2018, data05197a.tar
>> Sun Mar 04 04:45:01 CET 2018, data05185a.tar
>> Sun Mar 04 04:38:20 CET 2018, data05175a.tar
>> Sun Mar 04 04:50:42 CET 2018, data05207a.tar
>> Sun Mar 04 04:40:31 CET 2018, data05178a.tar
>> Sun Mar 04 04:52:30 CET 2018, data05217a.tar
>> Sun Mar 04 04:46:56 CET 2018, data05189a.tar
>> Sun Mar 04 04:39:05 CET 2018, data05176a.tar
>> Sun Mar 04 04:35:12 CET 2018, data05170a.tar
>> Sun Mar 04 04:50:33 CET 2018, data05206a.tar
>> Sun Mar 04 04:47:22 CET 2018, data05190a.tar
>> Sun Mar 04 04:51:34 CET 2018, data05212a.tar
>> Sun Mar 04 04:54:25 CET 2018, data05225a.tar
>> Sun Mar 04 04:36:55 CET 2018, data05172a.tar
>> Sun Mar 04 04:50:20 CET 2018, data05205a.tar
>> Sun Mar 04 04:51:58 CET 2018, data05214a.tar
>> Mon Mar 05 04:26:41 CET 2018, data05231a.tar
>> Mon Mar 05 04:19:14 CET 2018, data05230a.tar
>> Sun Mar 04 04:53:43 CET 2018, data05223a.tar
>> Sun Mar 04 04:44:31 CET 2018, data05184a.tar
>> Sun Mar 04 04:47:53 CET 2018, data05192a.tar
>> Sun Mar 04 04:39:47 CET 2018, data05177a.tar
>> Sun Mar 04 04:48:17 CET 2018, data05194a.tar
>> Sun Mar 04 04:52:09 CET 2018, data05215a.tar
>> Sun Mar 04 04:48:05 CET 2018, data05193a.tar
>> Sun Mar 04 04:52:52 CET 2018, data05219a.tar
>> Sun Mar 04 04:53:20 CET 2018, data05221a.tar
>> Sun Mar 04 04:50:52 CET 2018, data05208a.tar
>> Sun Mar 04 04:46:07 CET 2018, data05187a.tar
>> Sun Mar 04 04:51:12 CET 2018, data05210a.tar
>> Sun Mar 04 04:56:04 CET 2018, gc.log
>> Sun Mar 04 04:33:34 CET 2018, data05167a.tar
>> Sun Mar 04 04:41:47 CET 2018, data05180a.tar
>> Sun Mar 04 04:51:02 CET 2018, data05209a.tar
>> size 17.1 GB (17094875037 bytes)
>> -> compacting
>> 04:26:46.520 [main] WARN o.a.jackrabbit.oak.segment.Compactor - Deprecated argument 'compress-interval', please use 'oak.segment.compaction.cacheSize' instead.
>> -> cleaning up
>> -> writing new journal.log: bf675b66-e8e6-4bf4-a982-dc7316323e9c:859 root 1520222043417
>>
>> after
>> Mon Mar 05 04:51:23 CET 2018, data05285a.tar
>> Mon Mar 05 04:36:53 CET 2018, data05244a.tar
>> Mon Mar 05 04:48:54 CET 2018, data05276a.tar
>> Mon Mar 05 04:43:10 CET 2018, data05254a.tar
>> Mon Mar 05 04:53:55 CET 2018, data05291a.tar
>> Mon Mar 05 04:48:31 CET 2018, data05274a.tar
>> Mon Mar 05 04:50:47 CET 2018, data05283a.tar
>> Mon Mar 05 04:40:27 CET 2018, data05249a.tar
>> Mon Mar 05 04:52:22 CET 2018, data05288a.tar
>> Mon Mar 05 04:49:30 CET 2018, data05278a.tar
>> Mon Mar 05 04:28:56 CET 2018, data05233a.tar
>> Mon Mar 05 04:31:18 CET 2018, data05236a.tar
>> Mon Mar 05 04:52:02 CET 2018, data05287a.tar
>> Tue Jan 16 06:25:35 CET 2018, data01538a.tar.bak
>> Mon Mar 05 04:47:51 CET 2018, data05272a.tar
>> Mon Mar 05 04:36:02 CET 2018, data05243a.tar
>> Mon Mar 05 04:35:14 CET 2018, data05242a.tar
>> Mon Mar 05 04:42:46 CET 2018, data05253a.tar
>> Mon Mar 05 04:44:55 CET 2018, data05260a.tar
>> Mon Mar 05 04:39:07 CET 2018, data05247a.tar
>> Mon Mar 05 04:46:50 CET 2018, data05268a.tar
>> Mon Mar 05 04:44:06 CET 2018, data05256a.tar
>> Mon Mar 05 04:45:33 CET 2018, data05263a.tar
>> Mon Mar 05 04:53:22 CET 2018, data05290a.tar
>> Thu Dec 14 07:14:54 CET 2017, repo.lock
>> Mon Mar 05 04:50:19 CET 2018, data05281a.tar
>> Mon Mar 05 04:49:13 CET 2018, data05277a.tar
>> Mon Mar 05 04:45:51 CET 2018, data05264a.tar
>> Mon Mar 05 04:33:40 CET 2018, data05240a.tar
>> Mon Mar 05 04:46:08 CET 2018, data05265a.tar
>> Mon Mar 05 04:32:55 CET 2018, data05239a.tar
>> Mon Mar 05 04:41:04 CET 2018, data05250a.tar
>> Fri Mar 02 04:57:25 CET 2018, data05022a.tar.2.bak
>> Mon Mar 05 04:45:18 CET 2018, data05262a.tar
>> Mon Mar 05 04:30:20 CET 2018, data05235a.tar
>> Mon Mar 05 04:44:44 CET 2018, data05259a.tar
>> Mon Mar 05 04:51:43 CET 2018, data05286a.tar
>> Mon Mar 05 04:54:03 CET 2018, journal.log
>> Fri Mar 02 04:57:24 CET 2018, data05022a.tar.bak
>> Mon Mar 05 04:32:32 CET 2018, data05238a.tar
>> Mon Mar 05 04:28:18 CET 2018, data05232a.tar
>> Mon Mar 05 04:45:07 CET 2018, data05261a.tar
>> Mon Mar 05 04:47:38 CET 2018, data05271a.tar
>> Mon Mar 05 04:49:48 CET 2018, data05279a.tar
>> Mon Mar 05 04:53:56 CET 2018, manifest
>> Mon Mar 05 04:34:30 CET 2018, data05241a.tar
>> Mon Mar 05 04:46:20 CET 2018, data05266a.tar
>> Mon Mar 05 04:43:41 CET 2018, data05255a.tar
>> Mon Mar 05 04:38:21 CET 2018, data05246a.tar
>> Mon Mar 05 04:48:14 CET 2018, data05273a.tar
>> Mon Mar 05 04:52:50 CET 2018, data05289a.tar
>> Mon Mar 05 04:42:19 CET 2018, data05252a.tar
>> Mon Mar 05 04:41:47 CET 2018, data05251a.tar
>> Mon Mar 05 04:47:07 CET 2018, data05269a.tar
>> Mon Mar 05 04:50:34 CET 2018, data05282a.tar
>> Mon Mar 05 04:47:21 CET 2018, data05270a.tar
>> Mon Mar 05 04:39:50 CET 2018, data05248a.tar
>> Mon Mar 05 04:29:40 CET 2018, data05234a.tar
>> Mon Mar 05 04:44:19 CET 2018, data05257a.tar
>> Mon Mar 05 04:44:30 CET 2018, data05258a.tar
>> Mon Mar 05 04:32:13 CET 2018, data05237a.tar
>> Mon Mar 05 04:48:43 CET 2018, data05275a.tar
>> Mon Mar 05 04:50:05 CET 2018, data05280a.tar
>> Mon Mar 05 04:37:30 CET 2018, data05245a.tar
>> Mon Mar 05 04:51:02 CET 2018, data05284a.tar
>> Mon Mar 05 04:46:33 CET 2018, data05267a.tar
>> Mon Mar 05 04:54:03 CET 2018, gc.log
>> size 16.1 GB (16081145586 bytes)
>> removed files [data05228a.tar, data05222a.tar, data05167a.tar, data05184a.tar, data05214a.tar, data05192a.tar, data05200a.tar, data05198a.tar, data05183a.tar, data05189a.tar, data05205a.tar, data05175a.tar, data05230a.tar, data05206a.tar, data05191a.tar, data05197a.tar, data05174a.tar, data05177a.tar, data05188a.tar, data05171a.tar, data05182a.tar, data05169a.tar, data05224a.tar, data05227a.tar, data05221a.tar, data05213a.tar, data05193a.tar, data05185a.tar, data05190a.tar, data05216a.tar, data05219a.tar, data05196a.tar, data05199a.tar, data05195a.tar, data05211a.tar, data05187a.tar, data05203a.tar, data05208a.tar, data05186a.tar, data05172a.tar, data05180a.tar, data05202a.tar, data05210a.tar, data05178a.tar, data05209a.tar, data05217a.tar, data05225a.tar, data05194a.tar, data05207a.tar, data05218a.tar, data05229a.tar, data05220a.tar, data05168a.tar, data05223a.tar, data05226a.tar, data05179a.tar, data05176a.tar, data05231a.tar, data05170a.tar, data05215a.tar, data05201a.tar, data05181a.tar, data05173a.tar, data05204a.tar, data05212a.tar]
>> added files [data05259a.tar, data05267a.tar, data05245a.tar, data05237a.tar, data05253a.tar, data05261a.tar, data05283a.tar, data05244a.tar, data05258a.tar, data05291a.tar, data05236a.tar, data05275a.tar, data05260a.tar, data05252a.tar, data05289a.tar, data05232a.tar, data05246a.tar, data05243a.tar, data05238a.tar, data05235a.tar, data05240a.tar, data05279a.tar, data05262a.tar, data05268a.tar, data05276a.tar, data05251a.tar, data05284a.tar, data05270a.tar, data05254a.tar, data05273a.tar, data05281a.tar, data05265a.tar, data05249a.tar, data05257a.tar, data05250a.tar, data05256a.tar, data05264a.tar, data05239a.tar, data05287a.tar, data05272a.tar, data05242a.tar, data05248a.tar, data05241a.tar, data05280a.tar, data05247a.tar, data05255a.tar, data05278a.tar, data05233a.tar, data05286a.tar, data05271a.tar, data05274a.tar, data05288a.tar, data05263a.tar, data05266a.tar, data05285a.tar, data05290a.tar, data05277a.tar, data05282a.tar, data05269a.tar, data05234a.tar]
>> Compaction succeeded in 27.29 min (1637s).
>>
>> ```
>>
>> Greets,
>> Roy
Re: Offline compaction - performance
Posted by Michael Dürig <mi...@gmail.com>.
Roy,
One thing that might help is upgrading to 1.6.6 to get the fix for
OAK-6642 /OAK-6110. With that fix you should be able to run within a
2GB heap and compaction times should go down.
Re. the bak files: from an operational POV it is safe to remove them.
Those files are kept in the case an instances in shut down in an non
clean way and needs to recover the most recent tar file that hasn't
been properly closed. There is up to 5 seconds worth of data that
might get lost in that process. The bak files *might* contain parts of
this data and it *might* be possible to recover *some* of it manually.
If this is not needed (I haven't seen a case where it was so far), you
can safely remove them.
Michael
.
On 5 March 2018 at 14:44, Roy Teeuwen <ro...@teeuwen.be> wrote:
> Hey guys,
>
> We have upgraded to Oak 1.6.5 (AEM 6.3) and we have noticed that the offline compaction is a lot slower than it was initially in AEM 6.1. While we had compaction of 5-15 mins for 15GB at first, it now takes half an hour. Is there any performance tweaking / parameters we can use to speed it up? Below you can find the script we use to do daily offline compaction and a log file of one of those times. Usually it takes away one GB per day. Seeing as we have the possibility to take down the system once a night, we don't have to solely rely on online compaction.
>
> Another question I have with this, would it be ok that we remove the .bak files automatically after the compaction has ran? Or why do they stay in the folder?
>
> Script:
> ```
> #!/bin/bash
>
> # AEM config
> AEM_PATH="/opt/aem/author"
> AEM_USER="aem-user"
> CRX_QUICKSTART_PATH="${AEM_PATH}/crx-quickstart"
> REPOSITORY_PATH="${CRX_QUICKSTART_PATH}/repository"
> SEGMENTSTORE_PATH="${REPOSITORY_PATH}/segmentstore"
> AEM_STATUS="${CRX_QUICKSTART_PATH}/bin/status"
> AEM_START="${CRX_QUICKSTART_PATH}/bin/start"
> AEM_STOP="${CRX_QUICKSTART_PATH}/bin/stop"
>
> # OAK Run config
> OAK_RUN_JAR="/opt/aem_dependencies/aem/oak-run-1.6.5.jar"
> HEAPSIZE="-Xmx12228M"
> UPDATE_LIMIT="-Dupdate.limit=10000"
> COMPRESS_INTERVAL="-Dcompress-interval=100000"
> COMPACTION_PROGRESS_LOG="-Dcompaction-progress-log=150000"
> MEMORY_MAPPED="-Dtar.memoryMapped=true"
> OAK_RUN_PARAMETERS="${HEAPSIZE} ${UPDATE_LIMIT} ${COMPRESS_INTERVAL} ${COMPACTION_PROGRESS_LOG} ${MEMORY_MAPPED}"
> OAK_RUN="java ${OAK_RUN_PARAMETERS} -jar ${OAK_RUN_JAR}"
>
> if ! [ "$(whoami)" = "${AEM_USER}" ]
> then
> echo "Error: Maintenance task needs to be run as AEM user (${AEM_USER})."
> exit 1
> fi
>
> echo "==> Starting maintenance ($(date))"
>
> echo "==> Make sure AEM is not running"
> if ${AEM_STATUS}
> then
> echo "==> Stopping AEM.."
> ${AEM_STOP}
> fi
>
> echo "==> Cleanup unrefrenced checkpoints"
> ${OAK_RUN} checkpoints ${SEGMENTSTORE_PATH} || exit 1
> ${OAK_RUN} checkpoints ${SEGMENTSTORE_PATH} rm-unreferenced || exit 1
>
> echo "==> Compact TAR segmentstore"
> ${OAK_RUN} compact ${SEGMENTSTORE_PATH} || exit 1
>
> echo "==> Starting AEM"
> ${AEM_START}
>
> echo "==> Maintenance completed ($(date))"
>
> exit 0
>
> ```
>
> Logs:
>
> ```
> ==> Stopping AEM..
> 05.03.2018 04:00:01.716 *INFO * [main] Setting sling.home=. (command line)
> 05.03.2018 04:00:01.743 *INFO * [main] Sent 'stop' to /127.0.0.1:45726: OK
> Application not running
> ==> Cleanup unrefrenced checkpoints
> Apache Jackrabbit Oak 1.6.5
> Checkpoints /opt/aem/author/crx-quickstart/repository/segmentstore
> - 3f22ad0f-3b78-4764-a697-c323f6d95180 created 2018-03-05 04:26:30.67 expires 2020-11-29 04:26:30.67
> - 5caca09a-9d6b-426d-8166-e967ca1fbf7e created 2018-03-05 04:26:30.567 expires 2020-11-29 04:26:30.567
> Found 2 checkpoints
> Apache Jackrabbit Oak 1.6.5
> Checkpoints /opt/aem/author/crx-quickstart/repository/segmentstore
> Referenced checkpoint from /:async@async is 3f22ad0f-3b78-4764-a697-c323f6d95180
> Referenced checkpoint from /:async@fulltext-async is 5caca09a-9d6b-426d-8166-e967ca1fbf7e
> Removed 0 checkpoints in 44ms.
> ==> Compact TAR segmentstore
> Apache Jackrabbit Oak 1.6.5
> Compacting /opt/aem/author/crx-quickstart/repository/segmentstore
> With default access mode
> before
> Sun Mar 04 04:48:27 CET 2018, data05195a.tar
> Sun Mar 04 04:51:46 CET 2018, data05213a.tar
> Sun Mar 04 04:37:16 CET 2018, data05173a.tar
> Sun Mar 04 04:51:22 CET 2018, data05211a.tar
> Sun Mar 04 04:43:13 CET 2018, data05182a.tar
> Sun Mar 04 04:55:57 CET 2018, data05226a.tar
> Mon Mar 05 03:00:57 CET 2018, data05229a.tar
> Sun Mar 04 04:36:03 CET 2018, data05171a.tar
> Sun Mar 04 04:52:42 CET 2018, data05218a.tar
> Sun Mar 04 04:34:42 CET 2018, data05169a.tar
> Tue Jan 16 06:25:35 CET 2018, data01538a.tar.bak
> Sun Mar 04 04:34:07 CET 2018, data05168a.tar
> Sun Mar 04 04:49:31 CET 2018, data05201a.tar
> Sun Mar 04 04:53:31 CET 2018, data05222a.tar
> Sun Mar 04 06:11:04 CET 2018, data05228a.tar
> Sun Mar 04 04:46:31 CET 2018, data05188a.tar
> Sun Mar 04 04:50:08 CET 2018, data05204a.tar
> Sun Mar 04 04:48:36 CET 2018, data05196a.tar
> Thu Dec 14 07:14:54 CET 2017, repo.lock
> Sun Mar 04 04:43:54 CET 2018, data05183a.tar
> Sun Mar 04 04:49:08 CET 2018, data05199a.tar
> Sun Mar 04 04:49:43 CET 2018, data05202a.tar
> Fri Mar 02 04:57:25 CET 2018, data05022a.tar.2.bak
> Sun Mar 04 04:41:19 CET 2018, data05179a.tar
> Mon Mar 05 04:26:34 CET 2018, journal.log
> Fri Mar 02 04:57:24 CET 2018, data05022a.tar.bak
> Sun Mar 04 04:53:55 CET 2018, data05224a.tar
> Sun Mar 04 04:49:55 CET 2018, data05203a.tar
> Sun Mar 04 04:53:03 CET 2018, data05220a.tar
> Mon Mar 05 04:26:45 CET 2018, manifest
> Sun Mar 04 04:49:20 CET 2018, data05200a.tar
> Sun Mar 04 04:42:30 CET 2018, data05181a.tar
> Sun Mar 04 04:45:37 CET 2018, data05186a.tar
> Sun Mar 04 04:55:57 CET 2018, data05227a.tar
> Sun Mar 04 04:48:59 CET 2018, data05198a.tar
> Sun Mar 04 04:37:39 CET 2018, data05174a.tar
> Sun Mar 04 04:52:19 CET 2018, data05216a.tar
> Sun Mar 04 04:47:43 CET 2018, data05191a.tar
> Sun Mar 04 04:48:47 CET 2018, data05197a.tar
> Sun Mar 04 04:45:01 CET 2018, data05185a.tar
> Sun Mar 04 04:38:20 CET 2018, data05175a.tar
> Sun Mar 04 04:50:42 CET 2018, data05207a.tar
> Sun Mar 04 04:40:31 CET 2018, data05178a.tar
> Sun Mar 04 04:52:30 CET 2018, data05217a.tar
> Sun Mar 04 04:46:56 CET 2018, data05189a.tar
> Sun Mar 04 04:39:05 CET 2018, data05176a.tar
> Sun Mar 04 04:35:12 CET 2018, data05170a.tar
> Sun Mar 04 04:50:33 CET 2018, data05206a.tar
> Sun Mar 04 04:47:22 CET 2018, data05190a.tar
> Sun Mar 04 04:51:34 CET 2018, data05212a.tar
> Sun Mar 04 04:54:25 CET 2018, data05225a.tar
> Sun Mar 04 04:36:55 CET 2018, data05172a.tar
> Sun Mar 04 04:50:20 CET 2018, data05205a.tar
> Sun Mar 04 04:51:58 CET 2018, data05214a.tar
> Mon Mar 05 04:26:41 CET 2018, data05231a.tar
> Mon Mar 05 04:19:14 CET 2018, data05230a.tar
> Sun Mar 04 04:53:43 CET 2018, data05223a.tar
> Sun Mar 04 04:44:31 CET 2018, data05184a.tar
> Sun Mar 04 04:47:53 CET 2018, data05192a.tar
> Sun Mar 04 04:39:47 CET 2018, data05177a.tar
> Sun Mar 04 04:48:17 CET 2018, data05194a.tar
> Sun Mar 04 04:52:09 CET 2018, data05215a.tar
> Sun Mar 04 04:48:05 CET 2018, data05193a.tar
> Sun Mar 04 04:52:52 CET 2018, data05219a.tar
> Sun Mar 04 04:53:20 CET 2018, data05221a.tar
> Sun Mar 04 04:50:52 CET 2018, data05208a.tar
> Sun Mar 04 04:46:07 CET 2018, data05187a.tar
> Sun Mar 04 04:51:12 CET 2018, data05210a.tar
> Sun Mar 04 04:56:04 CET 2018, gc.log
> Sun Mar 04 04:33:34 CET 2018, data05167a.tar
> Sun Mar 04 04:41:47 CET 2018, data05180a.tar
> Sun Mar 04 04:51:02 CET 2018, data05209a.tar
> size 17.1 GB (17094875037 bytes)
> -> compacting
> 04:26:46.520 [main] WARN o.a.jackrabbit.oak.segment.Compactor - Deprecated argument 'compress-interval', please use 'oak.segment.compaction.cacheSize' instead.
> -> cleaning up
> -> writing new journal.log: bf675b66-e8e6-4bf4-a982-dc7316323e9c:859 root 1520222043417
>
> after
> Mon Mar 05 04:51:23 CET 2018, data05285a.tar
> Mon Mar 05 04:36:53 CET 2018, data05244a.tar
> Mon Mar 05 04:48:54 CET 2018, data05276a.tar
> Mon Mar 05 04:43:10 CET 2018, data05254a.tar
> Mon Mar 05 04:53:55 CET 2018, data05291a.tar
> Mon Mar 05 04:48:31 CET 2018, data05274a.tar
> Mon Mar 05 04:50:47 CET 2018, data05283a.tar
> Mon Mar 05 04:40:27 CET 2018, data05249a.tar
> Mon Mar 05 04:52:22 CET 2018, data05288a.tar
> Mon Mar 05 04:49:30 CET 2018, data05278a.tar
> Mon Mar 05 04:28:56 CET 2018, data05233a.tar
> Mon Mar 05 04:31:18 CET 2018, data05236a.tar
> Mon Mar 05 04:52:02 CET 2018, data05287a.tar
> Tue Jan 16 06:25:35 CET 2018, data01538a.tar.bak
> Mon Mar 05 04:47:51 CET 2018, data05272a.tar
> Mon Mar 05 04:36:02 CET 2018, data05243a.tar
> Mon Mar 05 04:35:14 CET 2018, data05242a.tar
> Mon Mar 05 04:42:46 CET 2018, data05253a.tar
> Mon Mar 05 04:44:55 CET 2018, data05260a.tar
> Mon Mar 05 04:39:07 CET 2018, data05247a.tar
> Mon Mar 05 04:46:50 CET 2018, data05268a.tar
> Mon Mar 05 04:44:06 CET 2018, data05256a.tar
> Mon Mar 05 04:45:33 CET 2018, data05263a.tar
> Mon Mar 05 04:53:22 CET 2018, data05290a.tar
> Thu Dec 14 07:14:54 CET 2017, repo.lock
> Mon Mar 05 04:50:19 CET 2018, data05281a.tar
> Mon Mar 05 04:49:13 CET 2018, data05277a.tar
> Mon Mar 05 04:45:51 CET 2018, data05264a.tar
> Mon Mar 05 04:33:40 CET 2018, data05240a.tar
> Mon Mar 05 04:46:08 CET 2018, data05265a.tar
> Mon Mar 05 04:32:55 CET 2018, data05239a.tar
> Mon Mar 05 04:41:04 CET 2018, data05250a.tar
> Fri Mar 02 04:57:25 CET 2018, data05022a.tar.2.bak
> Mon Mar 05 04:45:18 CET 2018, data05262a.tar
> Mon Mar 05 04:30:20 CET 2018, data05235a.tar
> Mon Mar 05 04:44:44 CET 2018, data05259a.tar
> Mon Mar 05 04:51:43 CET 2018, data05286a.tar
> Mon Mar 05 04:54:03 CET 2018, journal.log
> Fri Mar 02 04:57:24 CET 2018, data05022a.tar.bak
> Mon Mar 05 04:32:32 CET 2018, data05238a.tar
> Mon Mar 05 04:28:18 CET 2018, data05232a.tar
> Mon Mar 05 04:45:07 CET 2018, data05261a.tar
> Mon Mar 05 04:47:38 CET 2018, data05271a.tar
> Mon Mar 05 04:49:48 CET 2018, data05279a.tar
> Mon Mar 05 04:53:56 CET 2018, manifest
> Mon Mar 05 04:34:30 CET 2018, data05241a.tar
> Mon Mar 05 04:46:20 CET 2018, data05266a.tar
> Mon Mar 05 04:43:41 CET 2018, data05255a.tar
> Mon Mar 05 04:38:21 CET 2018, data05246a.tar
> Mon Mar 05 04:48:14 CET 2018, data05273a.tar
> Mon Mar 05 04:52:50 CET 2018, data05289a.tar
> Mon Mar 05 04:42:19 CET 2018, data05252a.tar
> Mon Mar 05 04:41:47 CET 2018, data05251a.tar
> Mon Mar 05 04:47:07 CET 2018, data05269a.tar
> Mon Mar 05 04:50:34 CET 2018, data05282a.tar
> Mon Mar 05 04:47:21 CET 2018, data05270a.tar
> Mon Mar 05 04:39:50 CET 2018, data05248a.tar
> Mon Mar 05 04:29:40 CET 2018, data05234a.tar
> Mon Mar 05 04:44:19 CET 2018, data05257a.tar
> Mon Mar 05 04:44:30 CET 2018, data05258a.tar
> Mon Mar 05 04:32:13 CET 2018, data05237a.tar
> Mon Mar 05 04:48:43 CET 2018, data05275a.tar
> Mon Mar 05 04:50:05 CET 2018, data05280a.tar
> Mon Mar 05 04:37:30 CET 2018, data05245a.tar
> Mon Mar 05 04:51:02 CET 2018, data05284a.tar
> Mon Mar 05 04:46:33 CET 2018, data05267a.tar
> Mon Mar 05 04:54:03 CET 2018, gc.log
> size 16.1 GB (16081145586 bytes)
> removed files [data05228a.tar, data05222a.tar, data05167a.tar, data05184a.tar, data05214a.tar, data05192a.tar, data05200a.tar, data05198a.tar, data05183a.tar, data05189a.tar, data05205a.tar, data05175a.tar, data05230a.tar, data05206a.tar, data05191a.tar, data05197a.tar, data05174a.tar, data05177a.tar, data05188a.tar, data05171a.tar, data05182a.tar, data05169a.tar, data05224a.tar, data05227a.tar, data05221a.tar, data05213a.tar, data05193a.tar, data05185a.tar, data05190a.tar, data05216a.tar, data05219a.tar, data05196a.tar, data05199a.tar, data05195a.tar, data05211a.tar, data05187a.tar, data05203a.tar, data05208a.tar, data05186a.tar, data05172a.tar, data05180a.tar, data05202a.tar, data05210a.tar, data05178a.tar, data05209a.tar, data05217a.tar, data05225a.tar, data05194a.tar, data05207a.tar, data05218a.tar, data05229a.tar, data05220a.tar, data05168a.tar, data05223a.tar, data05226a.tar, data05179a.tar, data05176a.tar, data05231a.tar, data05170a.tar, data05215a.tar, data05201a.tar, data05181a.tar, data05173a.tar, data05204a.tar, data05212a.tar]
> added files [data05259a.tar, data05267a.tar, data05245a.tar, data05237a.tar, data05253a.tar, data05261a.tar, data05283a.tar, data05244a.tar, data05258a.tar, data05291a.tar, data05236a.tar, data05275a.tar, data05260a.tar, data05252a.tar, data05289a.tar, data05232a.tar, data05246a.tar, data05243a.tar, data05238a.tar, data05235a.tar, data05240a.tar, data05279a.tar, data05262a.tar, data05268a.tar, data05276a.tar, data05251a.tar, data05284a.tar, data05270a.tar, data05254a.tar, data05273a.tar, data05281a.tar, data05265a.tar, data05249a.tar, data05257a.tar, data05250a.tar, data05256a.tar, data05264a.tar, data05239a.tar, data05287a.tar, data05272a.tar, data05242a.tar, data05248a.tar, data05241a.tar, data05280a.tar, data05247a.tar, data05255a.tar, data05278a.tar, data05233a.tar, data05286a.tar, data05271a.tar, data05274a.tar, data05288a.tar, data05263a.tar, data05266a.tar, data05285a.tar, data05290a.tar, data05277a.tar, data05282a.tar, data05269a.tar, data05234a.tar]
> Compaction succeeded in 27.29 min (1637s).
>
> ```
>
> Greets,
> Roy