[ignite] branch master updated: IGNITE-18515 CDC: add documentation about metadata replication (#10481)

[ni...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

nizhikov pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ignite.git


The following commit(s) were added to refs/heads/master by this push:
     new 3864f18a72e IGNITE-18515 CDC: add documentation about metadata replication (#10481)
3864f18a72e is described below

commit 3864f18a72ea0e29c967509362654bc11328864c
Author: Ilya Shishkov <sh...@gmail.com>
AuthorDate: Mon Jan 16 15:59:05 2023 +0300

    IGNITE-18515 CDC: add documentation about metadata replication (#10481)
---
 .../change-data-capture-extensions.adoc             | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc b/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc
index b9e248cf053..736f754eb3f 100644
--- a/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc
+++ b/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc
@@ -25,6 +25,8 @@ link:https://github.com/apache/ignite-extensions/tree/master/modules/cdc-ext[Cha
 
 NOTE: For each cache replicated between clusters link:https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/version/CacheVersionConflictResolver.java[CacheVersionConflictResolver] should be defined.
 
+NOTE: All implementations of CDC replication support replication of link:https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/binary/BinaryType.html[BinaryTypes] and link:https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cdc/TypeMapping.html[TypeMappings]
+
 == Ignite to Java Thin Client CDC streamer
 This streamer starts link:thin-clients/java-thin-client[Java Thin Client] which connects to destination cluster.
 After connection is established, all changes captured by CDC will be replicated to destination cluster.
@@ -92,6 +94,8 @@ This way to replicate changes between clusters requires setting up two applicati
 
 NOTE: Instances of `ignite-cdc.sh` with configured streamer should be started on each server node of source cluster to capture all changes.
 
+IMPORTANT: CDC trough Kafka requires _metadata topic with the only one partition_ for sequential ordering guarantees.
+
 image:../../assets/images/integrations/CDC-ignite2kafka.svg[]
 
 === IgniteToKafkaCdcStreamer Configuration
@@ -101,8 +105,9 @@ image:../../assets/images/integrations/CDC-ignite2kafka.svg[]
 |Name |Description | Default value
 | `caches` | Set of cache names to replicate. | null
 | `kafkaProperties` | Kafka producer properties. | null
-| `topic` | Name of the Kafka topic. | null
-| `kafkaParts` | Number of Kafka topic partitions. | null
+| `topic` | Name of the Kafka topic for CDC events. | null
+| `kafkaParts` | Number of Kafka partitions in CDC events topic. | null
+| `metadataTopic` | Name of topic for replication of BinaryTypes and TypeMappings. | null
 | `onlyPrimary` | Flag to handle changes only on primary node. | `false`
 | `maxBatchSize` | Maximum size of concurrently produced Kafka records. When streamer reaches this number, it waits for Kafka acknowledgements, and then commits CDC offset. | `1024`
 | `kafkaRequestTimeout` | Kafka request timeout in milliseconds.  | `3000`
@@ -160,9 +165,11 @@ Kafka to Ignite configuration file should contain the following beans that will
 |===
 |Name |Description | Default value
 | `caches` | Set of cache names to replicate. | null
-| `topic` | Name of the Kafka topic. | null
-| `kafkaPartsFrom` | Lower Kafka partitions number (inclusive). | -1
-| `kafkaPartsTo` | Lower Kafka partitions number (exclusive). | -1
+| `topic` | Name of the Kafka topic for CDC events. | null
+| `kafkaPartsFrom` | Lower Kafka partitions number (inclusive) for CDC events topic. | -1
+| `kafkaPartsTo` | Lower Kafka partitions number (exclusive) for CDC events topic. | -1
+| `metadataTopic` | Name of topic for replication of BinaryTypes and TypeMappings. | null
+| `metadataConsumerGroup` | Group for `KafkaConsumer`, which polls from metadata topic | ignite-metadata-update-<kafkaPartsFrom>-<kafkaPartsTo>
 | `kafkaRequestTimeout` | Kafka request timeout in milliseconds.  | `3000`
 | `maxBatchSize` | Maximum number of events to be sent to destination cluster in a single batch. | 1024
 | `threadCount` | Count of threads to proceed consumers. Each thread poll records from dedicated partitions in round-robin manner. | 16
@@ -170,7 +177,7 @@ Kafka to Ignite configuration file should contain the following beans that will
 
 ==== Logging
 
-`kakfa-to-ignite.sh` uses the same logging configuration as the Ignite node does. The only difference is that the log is written in the "kafka-ignite-streamer.log" file.
+`kafka-to-ignite.sh` uses the same logging configuration as the Ignite node does. The only difference is that the log is written in the "kafka-ignite-streamer.log" file.
 
 == CacheVersionConflictResolver implementation
 
@@ -223,4 +230,4 @@ Configuration is done via Ignite node plugin:
         </property>
     </bean>
 </property>
-```
\ No newline at end of file
+```