You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ignite.apache.org by na...@apache.org on 2022/08/04 13:30:48 UTC
[ignite] branch master updated: IGNITE-17450 Documentation: thin client CDC streamer (#10181)
This is an automated email from the ASF dual-hosted git repository.
namelchev pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/master by this push:
new 4beb0b683e6 IGNITE-17450 Documentation: thin client CDC streamer (#10181)
4beb0b683e6 is described below
commit 4beb0b683e6049084bcdcd8811bac4d3b5a08602
Author: Nikita Amelchev <ns...@gmail.com>
AuthorDate: Thu Aug 4 16:30:40 2022 +0300
IGNITE-17450 Documentation: thin client CDC streamer (#10181)
---
.../change-data-capture-extensions.adoc | 35 ++++++++++++++++++++--
.../integrations/CDC-ignite2igniteClient.svg | 4 +++
2 files changed, 37 insertions(+), 2 deletions(-)
diff --git a/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc b/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc
index 1c63bd23083..fce656174b5 100644
--- a/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc
+++ b/docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc
@@ -19,11 +19,40 @@ WARNING: CDC is an experimental feature. API or design architecture might be cha
== Overview
link:https://github.com/apache/ignite-extensions/tree/master/modules/cdc-ext[Change Data Capture Extension] module provides two ways to set up cross cluster replication based on CDC.
+. link:https://github.com/apache/ignite-extensions/blob/master/modules/cdc-ext/src/main/java/org/apache/ignite/cdc/thin/IgniteToIgniteClientCdcStreamer.java[Ignite2IgniteClientCdcStreamer] - streams changes to destination cluster using link:thin-clients/java-thin-client[Java Thin Client].
. link:https://github.com/apache/ignite-extensions/blob/master/modules/cdc-ext/src/main/java/org/apache/ignite/cdc/IgniteToIgniteCdcStreamer.java[Ignite2IgniteCdcStreamer] - streams changes to destination cluster using client node.
. link:https://github.com/apache/ignite-extensions/blob/master/modules/cdc-ext/src/main/java/org/apache/ignite/cdc/kafka/IgniteToKafkaCdcStreamer.java[Ignite2KafkaCdcStreamer] combined with link:https://github.com/apache/ignite-extensions/blob/master/modules/cdc-ext/src/main/java/org/apache/ignite/cdc/kafka/KafkaToIgniteCdcStreamer.java[KafkaToIgniteCdcStreamer] streams changes to destination cluster using link:https://kafka.apache.org[Apache Kafka] as a transport.
NOTE: For each cache replicated between clusters link:https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/version/CacheVersionConflictResolver.java[CacheVersionConflictResolver] should be defined.
+== Ignite to Java Thin Client CDC streamer
+This streamer starts link:thin-clients/java-thin-client[Java Thin Client] which connects to destination cluster.
+After connection is established, all changes captured by CDC will be replicated to destination cluster.
+
+NOTE: Instances of `ignite-cdc.sh` with configured streamer should be started on each server node of source cluster to capture all changes.
+
+image:../../assets/images/integrations/CDC-ignite2igniteClient.svg[]
+
+=== Configuration
+
+[cols="20%,45%,35%",opts="header"]
+|===
+|Name |Description | Default value
+| `caches` | Set of cache names to replicate. | null
+| `destinationClientConfiguration` | Client configuration of thin client that will connect to destination cluster to replicate changes. | null
+| `onlyPrimary` | Flag to handle changes only on primary node. | `false`
+| `maxBatchSize` | Maximum number of events to be sent to destination cluster in a single batch. | 1024
+|===
+
+=== Metrics
+
+|===
+|Name |Description
+| `EventsCount` | Count of messages applied to destination cluster.
+| `LastEventTime` | Timestamp of last applied event.
+| `TypesCount` | Count of received binary types events.
+| `MappingsCount` | Count of received mappings events.
+|===
== Ignite to Ignite CDC streamer
This streamer starts client node which connects to destination cluster.
@@ -33,7 +62,7 @@ NOTE: Instances of `ignite-cdc.sh` with configured streamer should be started on
image:../../assets/images/integrations/CDC-ignite2ignite.svg[]
-== Configuration
+=== Configuration
[cols="20%,45%,35%",opts="header"]
|===
@@ -44,12 +73,14 @@ image:../../assets/images/integrations/CDC-ignite2ignite.svg[]
| `maxBatchSize` | Maximum number of events to be sent to destination cluster in a single batch. | 1024
|===
-== Metrics
+=== Metrics
|===
|Name |Description
| `EventsCount` | Count of messages applied to destination cluster.
| `LastEventTime` | Timestamp of last applied event.
+| `TypesCount` | Count of received binary types events.
+| `MappingsCount` | Count of received mappings events.
|===
== CDC replication using Kafka
diff --git a/docs/assets/images/integrations/CDC-ignite2igniteClient.svg b/docs/assets/images/integrations/CDC-ignite2igniteClient.svg
new file mode 100644
index 00000000000..3d5461e5137
--- /dev/null
+++ b/docs/assets/images/integrations/CDC-ignite2igniteClient.svg
@@ -0,0 +1,4 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Do not edit this file with editors other than diagrams.net -->
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="881px" height="461px" viewBox="-0.5 -0.5 881 461" content="<mxfile host="app.diagrams.net" modified="2022-08-04T13:11:56.372Z" agent="5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15" etag="rbKeBkZv9W-cBPy4L6nv" version="20.2.3" type="device"><diagram id=&qu [...]
\ No newline at end of file