You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "xtern (via GitHub)" <gi...@apache.org> on 2024/09/03 14:59:00 UTC

[PR] IGNITE-22712 [ignite-3]

xtern opened a new pull request, #4325:
URL: https://github.com/apache/ignite-3/pull/4325

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "AMashenkov (via GitHub)" <gi...@apache.org>.
AMashenkov commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1745690675


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.
+Therefore, when the metastorage group leader changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed

Review Comment:
   Low watermark can be changed too frequently, right?
   And the compaction process is not fast/lightweight, right?
   If both answers are 'yes', then... is there any back off mechanics?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753215993


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,
+   as well as a set of local replication groups that was used to calculate the local minimum.
+3. Coordinator determines global minimum required time.
+4. Using this time, the coordinator determines the version of the catalog up to which (inclusive) history can be trimmed.
+5. Based on the calculated catalog version, the coordinator determines the list of partitions 

Review Comment:
   Fixed, thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753210190


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published

Review Comment:
   thanks, fixed partially



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.

Review Comment:
   thanks, fixed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753187714


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.

Review Comment:
   Thanks, fixed.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed

Review Comment:
   Thanks, fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753190287


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin

Review Comment:
   Thanks, fixed partially



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1746663520


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.
+Therefore, when the metastorage group leader changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed

Review Comment:
   Added this into [IGNITE-23059](https://issues.apache.org/jira/browse/IGNITE-23059)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "IgGusev (via GitHub)" <gi...@apache.org>.
IgGusev commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753412008


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,
+   as well as a set of local replication groups that was used to calculate the local minimum.

Review Comment:
   Then i misunderstood the statement, my bad.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753483246


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 

Review Comment:
   Thank you, I changed it as you suggested.
   Sounds better and in my opinion the main meaning does not change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753183935


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)

Review Comment:
   Thanks, fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753182138


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 

Review Comment:
   thanks, but for me "to a humongous size " sounds better :sunglasses:



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1746600315


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.
+Therefore, when the metastorage group leader changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+When compaction is triggered, two parallel processes are started

Review Comment:
   Done, thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "AMashenkov (via GitHub)" <gi...@apache.org>.
AMashenkov commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1745680175


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest

Review Comment:
   Maybe, it is better to replace "the earliest active transaction begin timestamp" with another term like "minimal observable timestamp".
   Then say that this time is minimum timestamp from next:
   1. tx
   2. raft log recover 
   and so on.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1746578676


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.
+Therefore, when the metastorage group leader changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed

Review Comment:
   > Low watermark can be changed too frequently, right?
   it's 5 mins by default, but it's configurable :thinking: 
   
   >  is there any back off mechanics?
   there is no back off mechanics, only that the process can't start if previous iteration is not finished yet.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753235937


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 

Review Comment:
   :thinking:  thanks , but I think it's better to keep original :sunglasses: 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753233604


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update

Review Comment:
   Done, thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753180823


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 

Review Comment:
   For example, we have 3 versions of catalog
   
   v1 activation time 1
   v2 activation time 3
   v3 activation time 5
   
   and we have transaction with activation time 4
   
   "excluding" means that we can trim only v1, because transaction uses catalog v2



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753182138


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 

Review Comment:
   thanks, but for me "to a humongous size " sounds better :sunglasses:



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753211000


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,

Review Comment:
   thanks, fixed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1746578676


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.
+Therefore, when the metastorage group leader changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed

Review Comment:
   > Low watermark can be changed too frequently, right?
   
   interval is 5 mins by default, but it's configurable :thinking: 
   
   >  is there any back off mechanics?
   
   currently there is no back off mechanics, only that the process can't start if previous iteration is not finished yet.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "AMashenkov (via GitHub)" <gi...@apache.org>.
AMashenkov commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1745691628


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.
+Therefore, when the metastorage group leader changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+When compaction is triggered, two parallel processes are started

Review Comment:
   ```suggestion
   When compaction is triggered, two processes are started in parallel
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753181184


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,

Review Comment:
   Thanks, fixed.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 

Review Comment:
   Thanks, fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753211716


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,
+   as well as a set of local replication groups that was used to calculate the local minimum.
+3. Coordinator determines global minimum required time.

Review Comment:
   Thanks, fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "IgGusev (via GitHub)" <gi...@apache.org>.
IgGusev commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753415534


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 

Review Comment:
   Catalog compaction can be performed up to the highest version, while excluding those with activation time lower...



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753181459


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 

Review Comment:
   Thanks, fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753244621


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 

Review Comment:
   Different here means something like "distinctive in nature"



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1746539474


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.
+Therefore, when the metastorage group leader changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+When compaction is triggered, two parallel processes are started
+
+- The first process (let's call it "**replicas update**") updates all replication groups
+  with determined minimum begin time among all active RW transactions in the cluster.
+  This time will be used by compaction process (see below) to comply with the restriction
+  about raft log recovery.
+- The second one ("**compaction**") determines the minimum required version of the catalog
+  and performs compaction.
+
+![Replicas update](tech-notes/processes.png) 
+
+> **_NOTE_** Even though the compaction process depends on the result of the replicas
+update process, they are executed in parallel, so the compaction in the current iteration
+will see the result of the replicas update process obtained in one of the previous iterations.

Review Comment:
   Merged to previous paragraph.
   ```
   - The second one ("**compaction**") determines the minimum required version of the catalog
     and performs compaction. At each iteration, it uses the result of the replica update
     process calculated at one of the previous iterations.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753231053


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,
+   as well as a set of local replication groups that was used to calculate the local minimum.
+3. Coordinator determines global minimum required time.
+4. Using this time, the coordinator determines the version of the catalog up to which (inclusive) history can be trimmed.
+5. Based on the calculated catalog version, the coordinator determines the list of partitions 
+   required by it and compares actual replication groups distribution with what was received
+   from remote nodes, and current iteration will be aborted in the following cases:
+   1. the logical topology is missing some node required by catalog
+   2. some node is missing required replication group
+   3. calculated catalog version has an index that is still building
+   4. there is an active rebalance, which refer to the calculated (or below calculated) version of the catalog
+6. Perform catalog compaction.

Review Comment:
   Fixed, thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1746578676


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.
+Therefore, when the metastorage group leader changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed

Review Comment:
   > Low watermark can be changed too frequently, right?
   
   it's 5 mins by default, but it's configurable :thinking: 
   
   >  is there any back off mechanics?
   
   currently there is no back off mechanics, only that the process can't start if previous iteration is not finished yet.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "AMashenkov (via GitHub)" <gi...@apache.org>.
AMashenkov commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1745666400


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest

Review Comment:
   You may have few catalog versions v1 and v2 with activation time 1 and 3.
   A transaction with activation time of 2 will use catalog of version v1.
   So, v1 must not be compacted.
   
   I'd say we can compact catalog up to the highest version (excluding) which activation time is less or equal to the earliest active transaction begin timestamp.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "AMashenkov (via GitHub)" <gi...@apache.org>.
AMashenkov commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1745683720


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.

Review Comment:
   ```suggestion
   Compaction is performed from single node (compaction coordinator) which is the leader of the metastorage group for simplicity.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1746653418


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest

Review Comment:
   changed to
   ```
   1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
      or equal to the earliest active transaction begin timestamp.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753222667


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,
+   as well as a set of local replication groups that was used to calculate the local minimum.

Review Comment:
   Didn't get it. what "as well as TO the set" means here.
   We sending not only timestamp but also a set



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753182568


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.

Review Comment:
   Thanks, fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753204222


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.

Review Comment:
   Thanks, fixed.partially



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753187085


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance

Review Comment:
   Thanks, fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "AMashenkov (via GitHub)" <gi...@apache.org>.
AMashenkov commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1747309008


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.
+Therefore, when the metastorage group leader changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed

Review Comment:
   Let's just mention `low watermark` is triggered sporadically (every 5min by default).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "AMashenkov (via GitHub)" <gi...@apache.org>.
AMashenkov commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1745742445


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.
+Therefore, when the metastorage group leader changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+When compaction is triggered, two parallel processes are started
+
+- The first process (let's call it "**replicas update**") updates all replication groups
+  with determined minimum begin time among all active RW transactions in the cluster.
+  This time will be used by compaction process (see below) to comply with the restriction
+  about raft log recovery.
+- The second one ("**compaction**") determines the minimum required version of the catalog
+  and performs compaction.
+
+![Replicas update](tech-notes/processes.png) 
+
+> **_NOTE_** Even though the compaction process depends on the result of the replicas
+update process, they are executed in parallel, so the compaction in the current iteration
+will see the result of the replicas update process obtained in one of the previous iterations.

Review Comment:
   I think this should be merged into previous paragraph.
   
   If my understanding right, 
   the first process calculates minimal TX begin time, which will be used later on the next compaction triggering.
   the second process uses result (TX begin time) of the previous run of the first process.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1746540005


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java) 
+component, which is responsible for periodically performing catalog compaction ensuring that
+dropped versions of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog must not be compacted by version which activation time is greater than or equal to earliest
+   active transaction in the cluster.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node called compaction coordinator.
+To simplify the selection of the coordinator, it was decided to consider it to be the same node 
+as the leader of the metastorage group.

Review Comment:
   Done, thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern merged PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "IgGusev (via GitHub)" <gi...@apache.org>.
IgGusev commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1751803749


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 

Review Comment:
   Without context catalog module comes from nowhere. Its fine if people know what this is about, but bad if they do not. I suggest to add "to a separate module from the catalog module"



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,

Review Comment:
   This is already highlighted, we do not need NOTE. Or we need it to be NOTE: to separate from text.
   Has been moved -> was moved. Avoid perfect times if you can.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 

Review Comment:
   the log -> log file
   to a humongous size - extra space.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 

Review Comment:
   Probably replace different with other, if i understand correctly what is meant here.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 

Review Comment:
   to see more details -> for more information



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update

Review Comment:
   catalog update..., drop the for catalog references. It is not a specific catalog, just catalog.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 

Review Comment:
   What does excluding mean here? 
   I also do not understand "which activation time..." part.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader

Review Comment:
   Compaction is performed from a single node (compaction coordinator), that is also the metastorage group leader



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only

Review Comment:
   (becomes available to compaction process)



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)

Review Comment:
   Avoid perfect.
   `low watermark` changed (every 5 minutes by default)



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).

Review Comment:
   after checkpoint happens  (partition data is flushed to disk).



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,

Review Comment:
   Each node sends the calculated local minimum timestamp to the coordinator,



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,
+   as well as a set of local replication groups that was used to calculate the local minimum.
+3. Coordinator determines global minimum required time.
+4. Using this time, the coordinator determines the version of the catalog up to which (inclusive) history can be trimmed.
+5. Based on the calculated catalog version, the coordinator determines the list of partitions 

Review Comment:
   Based on the calculated catalog version, the coordinator determines the list of required partitions 



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,
+   as well as a set of local replication groups that was used to calculate the local minimum.
+3. Coordinator determines global minimum required time.
+4. Using this time, the coordinator determines the version of the catalog up to which (inclusive) history can be trimmed.
+5. Based on the calculated catalog version, the coordinator determines the list of partitions 
+   required by it and compares actual replication groups distribution with what was received
+   from remote nodes, and current iteration will be aborted in the following cases:
+   1. the logical topology is missing some node required by catalog
+   2. some node is missing required replication group
+   3. calculated catalog version has an index that is still building
+   4. there is an active rebalance, which refer to the calculated (or below calculated) version of the catalog
+6. Perform catalog compaction.

Review Comment:
   The node performs catalog compaction.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.

Review Comment:
   Do versions perform catalog compaction? Maybe something like this?
   Catalog must not be compacted for the version that may be necessary to replay the raft log during recovery.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until

Review Comment:
   Index building uses a specific catalog version. This version cannot be truncated until



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin

Review Comment:
   **Replicas update**. Updates all replication groups with preset minimum start
      time among all active read-write transactions in the cluster. After some time (see below for details),
      these timestamps are published and become available for the next phase.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance

Review Comment:
   Rebalance uses a specific catalog version.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.

Review Comment:
   metastore -> metastorage



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates

Review Comment:
   **Compaction**. By using the timestamps published on the previous stage, the coordinator calculates
      the minimum required version of the catalog and performs compaction.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed

Review Comment:
   Compaction coordinator changed



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.

Review Comment:
   to determine the minimum start time among all local active read-write transactions and sends it to coordinator.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published

Review Comment:
   Using the introduced `RaftGroupStateProvider` method to determine the minimum time among all published



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.

Review Comment:
   The node selects...



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,
+   as well as a set of local replication groups that was used to calculate the local minimum.

Review Comment:
   as well as to the set of local replication groups that was used to calculate the local minimum.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,
+   as well as a set of local replication groups that was used to calculate the local minimum.
+3. Coordinator determines global minimum required time.

Review Comment:
   The coordinator determines global minimum required time.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753183654


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until

Review Comment:
   Thanks, fixed.



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader

Review Comment:
   Thanks, fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753199259


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates

Review Comment:
   Thanks, fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753205116


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only

Review Comment:
   thanks, fixed



##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).

Review Comment:
   thanks, fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-22712 Describe catalog compaction in README.md [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #4325:
URL: https://github.com/apache/ignite-3/pull/4325#discussion_r1753222667


##########
modules/catalog-compaction/README.md:
##########
@@ -1 +1,101 @@
 # Catalog compaction module
+
+> **_NOTE_** Compaction has been moved to a separate module to eliminate circular dependencies,
+as it requires some components that may themselves depend on the catalog module. Please 
+refer to the catalog's module [readme](../catalog/README.md) to see more details about 
+catalog service and update log.
+
+## Overview
+
+During schema changes, the catalog update log stores incremental updates. Each update
+increases the catalog version. Over time, the log may grow to a humongous  size. To 
+address this, snapshotting was introduced to UpdateLog. Snapshotting means replacing 
+incremental updates with a snapshot.
+
+But different components can refer to a specific version of the catalog. Until they 
+finish their work with this version, it cannot be truncated.
+
+This module introduces [CatalogCompactionRunner](src/main/java/org/apache/ignite/internal/catalog/compaction/CatalogCompactionRunner.java)
+component. This component handles periodical catalog compaction, ensuring that dropped  versions
+of the catalog are no longer needed by any component in the cluster.
+
+## Compaction restrictions
+
+1. Catalog compaction can be performed up to the highest version (excluding) which activation time is less 
+   or equal to the earliest active transaction begin timestamp.
+2. Catalog must not be compacted by version which can be required to replay the raft log during recovery.
+3. Index building is stick with a specific catalog version. This version cannot be truncated until
+   the index build is complete.
+4. Rebalance is stick with a specific catalog version. This version cannot be truncated until the rebalance
+   is complete.
+
+## Coordinator
+
+Compaction is performed from single node (compaction coordinator) which is the leader
+of the metastorage group for simplicity. Therefore, when the metastorage group leader
+changes, the compaction coordinator also changes.
+
+The [ElectionListener](../metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/ElectionListener.java)
+interface was introduced to listen for metastore leader elections.
+
+## Triggering factors
+
+The process is initiated by one of the following events:
+
+1. `low watermark` has been changed (every 5 minutes by default)
+2. the compaction coordinator has been changed
+
+## Overall process description
+
+Catalog compaction consists of two main stages:
+
+1. **Replicas update**. Updates all replication groups with determined minimum begin
+   time among all active RW transactions in the cluster. After some time (see below for details)
+   these timestamps are published and become available for the next phase.
+
+2. **Compaction**. Using the timestamps published on the previous stage calculates
+   the minimum required version of the catalog and perform compaction.
+
+Publishing timestamps can take a long time, and the success of compaction depends on more
+than just these timestamps. That's why both stages run in parallel. Thus, the compaction
+stage uses the result of the replicas update calculated at one of the previous iterations.
+To minimize the number of network requests, both processes run simultaneously and use a common
+[request](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionMinimumTimesRequest.java)
+to collect timestamps from the entire cluster in one round trip.
+
+### Replicas update stage
+
+![Replicas update](tech-notes/replicas-update.png)
+
+This stage consists of the following steps:
+
+1. Each node uses [ActiveLocalTxMinimumBeginTimeProvider](../transactions/src/main/java/org/apache/ignite/internal/tx/ActiveLocalTxMinimumBeginTimeProvider.java)
+   to determine the minimum begin time among all local active RW transactions and sends it to coordinator.
+2. Coordinator calculates global minimum and sends it to all nodes using [CatalogCompactionPrepareUpdateTxBeginTimeMessage](src/main/java/org/apache/ignite/internal/catalog/compaction/message/CatalogCompactionPrepareUpdateTxBeginTimeMessage.java).
+3. Each node stores this time within replication groups for which the local node is the leader  
+   (using [UpdateMinimumActiveTxBeginTimeReplicaRequest](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/replication/UpdateMinimumActiveTxBeginTimeReplicaRequest.java)
+   and [UpdateMinimumActiveTxBeginTimeCommand](../partition-replicator/src/main/java/org/apache/ignite/internal/partition/replicator/network/command/UpdateMinimumActiveTxBeginTimeCommand.java). 
+4. This timestamp (let's call it `minTxTime`) is published (become available to compaction process) only
+   after checkpoint (flushing partition data to disk).
+
+### Compaction stage
+
+![Replicas update](tech-notes/compaction.png)
+
+1. Each node determines the local minimum required time, this consists of the following steps:
+   1. Using introduced `RaftGroupStateProvider` determine minimum time among all published
+      timestamps (`minTxTime`) in local replication groups.
+   2. If `minTxTime` is not published yet, the current iteration of compaction is aborted.
+   3. Select minimum between determined minimum `minTxTime` and current `low watermark`.
+2. Each node sends to the coordinator the calculated local minimum timestamp,
+   as well as a set of local replication groups that was used to calculate the local minimum.

Review Comment:
   Didn't get it. what "as well as TO the set" mean here.
   We sending not only timestamp but also a set



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org