You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by or...@apache.org on 2024/02/19 11:09:26 UTC

(camel) branch main updated (7a778a6a7c0 -> d04c0e97443)

This is an automated email from the ASF dual-hosted git repository.

orpiske pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git


    from 7a778a6a7c0 Bump sshd-version from 2.12.0 to 2.12.1
     new ae9e31fddb2 CAMEL-20410: documentation fixes for camel-dataset
     new 33b49ac1cca CAMEL-20410: documentation fixes for camel-debezium-db2
     new d04c0e97443 CAMEL-20410: documentation fixes for camel-debezium-mongodb

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../src/main/docs/dataset-component.adoc           | 73 ++++++++++------------
 .../src/main/docs/debezium-db2-component.adoc      | 32 +++++++---
 .../src/main/docs/debezium-mongodb-component.adoc  | 42 +++++++++----
 3 files changed, 87 insertions(+), 60 deletions(-)


(camel) 02/03: CAMEL-20410: documentation fixes for camel-debezium-db2

Posted by or...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

orpiske pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git

commit 33b49ac1cca104a2a3b75d62e4c8e6fca9d281fc
Author: Otavio Rodolfo Piske <an...@gmail.com>
AuthorDate: Mon Feb 19 11:15:18 2024 +0100

    CAMEL-20410: documentation fixes for camel-debezium-db2
    
    - Fixed samples
    - Fixed grammar and typos
    - Fixed punctuation
    - Added and/or fixed links
---
 .../src/main/docs/debezium-db2-component.adoc      | 32 ++++++++++++++++------
 1 file changed, 23 insertions(+), 9 deletions(-)

diff --git a/components/camel-debezium/camel-debezium-db2/src/main/docs/debezium-db2-component.adoc b/components/camel-debezium/camel-debezium-db2/src/main/docs/debezium-db2-component.adoc
index 1054b451d50..b8543f0cdc5 100644
--- a/components/camel-debezium/camel-debezium-db2/src/main/docs/debezium-db2-component.adoc
+++ b/components/camel-debezium/camel-debezium-db2/src/main/docs/debezium-db2-component.adoc
@@ -18,9 +18,19 @@
 
 The Debezium db2 component is wrapper around https://debezium.io/[Debezium] using https://debezium.io/documentation/reference/1.8/operations/embedded.html[Debezium Engine], which enables Change Data Capture from db2 database using Debezium without the need for Kafka or Kafka Connect.
 
-*Note on handling failures:* Per https://debezium.io/documentation/reference/1.9/development/engine.html#_handling_failures[Debezium Embedded Engine] documentation, the engines is actively recording source offsets and periodically flushes these offsets to a persistent storage, so when the application is restarted or crashed, the engine will resume from the last recorded offset.
-Thus, at normal operation, your downstream routes will receive each event exactly once, however in case of an application crash (not having a graceful shutdown), the application will resume from the last recorded offset,
-which may result in receiving duplicate events immediately after the restart. Therefore, your downstream routes should be tolerant enough of such case and deduplicate events if needed.
+[NOTE]
+====
+*Note on handling failures:*
+per https://debezium.io/documentation/reference/1.9/development/engine.html#_handling_failures[Debezium Embedded Engine] documentation,
+the engines are actively recording source offsets and periodically flush these offsets to a persistent storage.
+Therefore, when the application is restarted or crashed, the engine will resume from the last recorded offset.
+This means that, at normal operation, your downstream routes will receive each event exactly once.
+However, in case of an application crash (not having a graceful shutdown),
+the application will resume from the last recorded offset,
+which may result in receiving duplicate events immediately after the restart.
+Therefore, your downstream routes should be tolerant enough of such a case and deduplicate events if needed.
+====
+
 
 Maven users will need to add the following dependency to their `pom.xml`
 for this component.
@@ -70,7 +80,7 @@ The message body if is not `null` (in case of tombstones), it contains the state
 
 === Consuming events
 
-Here is a very simple route that you can use in order to listen to Debezium events from db2 connector.
+Here is a basic route that you can use to listen to Debezium events from the db2 connector:
 
 [source,java]
 ----
@@ -87,10 +97,10 @@ from("debezium-db2:dbz-test-1?offsetStorageFileName=/usr/offset-file-1.dat&datab
 By default, the component will emit the events in the body and `CamelDebeziumBefore` header as https://kafka.apache.org/22/javadoc/org/apache/kafka/connect/data/Struct.html[`Struct`] data type, the reasoning behind this, is to perceive the schema information in case is needed.
 However, the component as well contains a xref:manual::type-converter.adoc[Type Converter] that converts
 from default output type of https://kafka.apache.org/22/javadoc/org/apache/kafka/connect/data/Struct.html[`Struct`] to `Map` in order to leverage Camel's rich xref:manual::data-format.adoc[Data Format] types which many of them work out of box with `Map` data type.
-To use it, you can either add `Map.class` type when you access the message e.g: `exchange.getIn().getBody(Map.class)`, or you can convert the body always to `Map` from the route builder by adding `.convertBodyTo(Map.class)` to your Camel Route DSL after `from` statement.
+To use it, you can either add `Map.class` type when you access the message (e.g., : `exchange.getIn().getBody(Map.class)`), or you can convert the body always to `Map` from the route builder by adding `.convertBodyTo(Map.class)` to your Camel Route DSL after `from` statement.
 
-We mentioned above about the schema, which can be used in case you need to perform advance data transformation and the schema is needed for that. If you choose not to convert your body to `Map`,
-you can obtain the schema information as https://kafka.apache.org/22/javadoc/org/apache/kafka/connect/data/Schema.html[`Schema`] type from `Struct` like this:
+We mentioned above the schema, which can be used in case you need to perform advance data transformation and the schema is needed for that.
+If you choose not to convert your body to `Map`, you can obtain the schema information as https://kafka.apache.org/22/javadoc/org/apache/kafka/connect/data/Schema.html[`Schema`] type from `Struct` like this:
 
 [source,java]
 ----
@@ -106,7 +116,11 @@ from("debezium-db2:[name]?[options]])
     });
 ----
 
-*Important Note:* This component is a thin wrapper around Debezium Engine as mentioned, therefore before using this component in production, you need to understand how Debezium works and how configurations can reflect the expected behavior, especially in regards to https://debezium.io/documentation/reference/1.8/operations/embedded.html#_handling_failures[handling failures].
-
+[IMPORTANT]
+====
+This component is a thin wrapper around Debezium Engine as mentioned.
+Therefore, before using this component in production, you need to understand how Debezium works and how configurations can reflect the expected behavior.
+This is especially true in regard to https://debezium.io/documentation/reference/1.8/operations/embedded.html#_handling_failures[handling failures].
+====
 
 include::spring-boot:partial$starter.adoc[]


(camel) 03/03: CAMEL-20410: documentation fixes for camel-debezium-mongodb

Posted by or...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

orpiske pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git

commit d04c0e97443f36c423de249bac4fb9122b5a1b7c
Author: Otavio Rodolfo Piske <an...@gmail.com>
AuthorDate: Mon Feb 19 11:34:59 2024 +0100

    CAMEL-20410: documentation fixes for camel-debezium-mongodb
    
    - Fixed samples
    - Fixed grammar and typos
    - Fixed punctuation
    - Added and/or fixed links
---
 .../src/main/docs/debezium-mongodb-component.adoc  | 42 +++++++++++++++-------
 1 file changed, 30 insertions(+), 12 deletions(-)

diff --git a/components/camel-debezium/camel-debezium-mongodb/src/main/docs/debezium-mongodb-component.adoc b/components/camel-debezium/camel-debezium-mongodb/src/main/docs/debezium-mongodb-component.adoc
index 967f27c5135..d89b1f08379 100644
--- a/components/camel-debezium/camel-debezium-mongodb/src/main/docs/debezium-mongodb-component.adoc
+++ b/components/camel-debezium/camel-debezium-mongodb/src/main/docs/debezium-mongodb-component.adoc
@@ -17,13 +17,25 @@
 
 The Debezium MongoDB component is wrapper around https://debezium.io/[Debezium] using https://debezium.io/documentation/reference/1.9/development/engine.html[Debezium Engine], which enables Change Data Capture from MongoDB database using Debezium without the need for Kafka or Kafka Connect.
 
-*Note:* The Debezium MongoDB connector uses MongoDB’s oplog to capture the changes, so the connector works only with MongoDB replica sets or with sharded clusters where each shard is a separate replica set, therefore you will need to have your MongoDB instance running either in replica set mode
-or sharded clusters mode.
-
-*Note on handling failures:* Per https://debezium.io/documentation/reference/1.9/development/engine.html#_handling_failures[Debezium Embedded Engine] documentation, the engines is actively recording source offsets and periodically flushes these offsets to a persistent storage, so when the application is restarted or crashed, the engine will resume from the last recorded offset.
-Thus, at normal operation, your downstream routes will receive each event exactly once, however in case of an application crash (not having a graceful shutdown), the application will resume from the last recorded offset,
-which may result in receiving duplicate events immediately after the restart. Therefore, your downstream routes should be tolerant enough of such case and deduplicate events if needed.
-
+[NOTE]
+====
+The Debezium MongoDB connector uses MongoDB’s oplog to capture the changes.
+The connector works only with the MongoDB replica sets or with sharded clusters, where each shard is a separate replica set.
+Therefore, you will need to have your MongoDB instance running either in replica set mode or sharded clusters mode.
+====
+
+[NOTE]
+====
+*Note on handling failures:*
+per https://debezium.io/documentation/reference/1.9/development/engine.html#_handling_failures[Debezium Embedded Engine] documentation,
+the engines are actively recording source offsets and periodically flush these offsets to a persistent storage.
+Therefore, when the application is restarted or crashed, the engine will resume from the last recorded offset.
+This means that, at normal operation, your downstream routes will receive each event exactly once.
+However, in case of an application crash (not having a graceful shutdown),
+the application will resume from the last recorded offset,
+which may result in receiving duplicate events immediately after the restart.
+Therefore, your downstream routes should be tolerant enough of such a case and deduplicate events if needed.
+====
 Maven users will need to add the following dependency to their `pom.xml`
 for this component.
 
@@ -68,13 +80,13 @@ include::partial$component-endpoint-headers.adoc[]
 *Note*: Debezium Mongodb uses MongoDB’s oplog to populate the CDC events, the update events in MongoDB’s oplog don’t have the before or after states of the changed document, so there’s no way for the Debezium connector to provide this information, therefore header key `CamelDebeziumBefore` is not available in this component.
 
 == Message body
-The message body if is not `null` (in case of tombstones), it contains the state of the row after the event occurred as `String` JSON format and you can unmarchal using Camel JSON Data Format.
+The message body if is not `null` (in case of tombstones), it contains the state of the row after the event occurred as `String` JSON format, and you can unmarshal using Camel JSON Data Format.
 
 == Samples
 
 === Consuming events
 
-Here is a very simple route that you can use in order to listen to Debezium events from MongoDB connector.
+Here is a basic route that you can use to listen to Debezium events from MongoDB connector:
 
 [source,java]
 ----
@@ -93,9 +105,15 @@ from("debezium-mongodb:dbz-test-1?offsetStorageFileName=/usr/offset-file-1.dat&m
     .end();
 ----
 
-By default, the component will emit the events in the body String JSON format in case of `u`, `c` or `r` operations, this can be easily converted to JSON using Camel JSON Data Format e.g: `.unmarshal().json()` like the above example. In case of operation `d`, the body will be `null`.
-
-*Important Note:* This component is a thin wrapper around Debezium Engine as mentioned, therefore before using this component in production, you need to understand how Debezium works and how configurations can reflect the expected behavior, especially in regards to https://debezium.io/documentation/reference/1.9/development/engine.html#_handling_failures[handling failures].
+By default, the component will emit the events in the body String JSON format in case of `u`, `c` or `r` operations.
+This can be easily converted to JSON using Camel JSON Data Format e.g.: `.unmarshal().json()` like the above example.
+In case of operation `d`, the body will be `null`.
 
+[IMPORTANT]
+====
+This component is a thin wrapper around Debezium Engine as mentioned.
+Therefore, before using this component in production, you need to understand how Debezium works and how configurations can reflect the expected behavior.
+This is especially true in regard to https://debezium.io/documentation/reference/1.8/operations/embedded.html#_handling_failures[handling failures].
+====
 
 include::spring-boot:partial$starter.adoc[]


(camel) 01/03: CAMEL-20410: documentation fixes for camel-dataset

Posted by or...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

orpiske pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git

commit ae9e31fddb2cafe3bec95f36f805d04ab0960894
Author: Otavio Rodolfo Piske <an...@gmail.com>
AuthorDate: Mon Feb 19 11:02:32 2024 +0100

    CAMEL-20410: documentation fixes for camel-dataset
    
    - Fixed samples
    - Fixed grammar and typos
    - Fixed punctuation
    - Added and/or fixed links
---
 .../src/main/docs/dataset-component.adoc           | 73 ++++++++++------------
 1 file changed, 34 insertions(+), 39 deletions(-)

diff --git a/components/camel-dataset/src/main/docs/dataset-component.adoc b/components/camel-dataset/src/main/docs/dataset-component.adoc
index e531f1bf72e..2bb2d3ebf6f 100644
--- a/components/camel-dataset/src/main/docs/dataset-component.adoc
+++ b/components/camel-dataset/src/main/docs/dataset-component.adoc
@@ -15,24 +15,18 @@
 
 *{component-header}*
 
-Testing of distributed and asynchronous processing is
-notoriously difficult. The xref:mock-component.adoc[Mock],
-xref:dataset-component.adoc[DataSet], and xref:dataset-test-component.adoc[DataSet Test]
-endpoints work great with the
-Camel Testing Framework to simplify your unit and
-integration testing using
-xref:eips:enterprise-integration-patterns.adoc[Enterprise Integration
-Patterns] and Camel's large range of Components
+Testing of distributed and asynchronous processing is notoriously challenging.
+The xref:mock-component.adoc[Mock], xref:dataset-component.adoc[DataSet], and xref:dataset-test-component.adoc[DataSet Test]
+endpoints work with the Camel Testing Framework to simplify your unit and integration testing using
+xref:eips:enterprise-integration-patterns.adoc[Enterprise Integration Patterns] and Camel's large range of Components
 together with the powerful Bean Integration.
 
-The DataSet component provides a mechanism to easily perform load & soak
-testing of your system. It works by allowing you to create
-https://www.javadoc.io/doc/org.apache.camel/camel-dataset/current/org/apache/camel/component/dataset/DataSet.html[DataSet
-instances] both as a source of messages and as a way to assert that the
-data set is received.
+The DataSet component provides a mechanism to easily perform load & soak testing of your system.
+It works by allowing you to create
+https://www.javadoc.io/doc/org.apache.camel/camel-dataset/current/org/apache/camel/component/dataset/DataSet.html[DataSet instances]
+both as a source of messages and as a way to assert that the data set is received.
 
-Camel will use the xref:log-component.adoc[throughput logger] when sending
-dataset's.
+Camel will use the xref:log-component.adoc[throughput logger] when sending the dataset.
 
 == URI format
 
@@ -41,13 +35,13 @@ dataset:name[?options]
 ----
 
 Where *name* is used to find the
-https://www.javadoc.io/doc/org.apache.camel/camel-dataset/current/org/apache/camel/component/dataset/DataSet.html[DataSet
-instance] in the Registry
+https://www.javadoc.io/doc/org.apache.camel/camel-dataset/current/org/apache/camel/component/dataset/DataSet.html[DataSet instance]
+in the Registry
 
 Camel ships with a support implementation of
 `org.apache.camel.component.dataset.DataSet`, the
 `org.apache.camel.component.dataset.DataSetSupport` class, that can be
-used as a base for implementing your own DataSet. Camel also ships with
+used as a base for implementing your own data set. Camel also ships with
 some implementations that can be used for testing:
  `org.apache.camel.component.dataset.SimpleDataSet`, `org.apache.camel.component.dataset.ListDataSet`
 and `org.apache.camel.component.dataset.FileDataSet`, all of which
@@ -73,9 +67,8 @@ include::partial$component-endpoint-headers.adoc[]
 
 == Configuring DataSet
 
-Camel will lookup in the Registry for a bean
-implementing the DataSet interface. So you can register your own DataSet
-as:
+Camel will look up in the Registry for a bean implementing the `DataSet` interface.
+So you can register your own data set as:
 
 [source,xml]
 ----
@@ -99,17 +92,17 @@ from("activemq:SomeQueue").to("dataset:foo");
 ----
 
 The above would look in the Registry to find the
-*foo* DataSet instance which is used to create the messages.
+`foo` `DataSet` instance which is used to create the messages.
 
-Then you create a DataSet implementation, such as using the
+Then you create a `DataSet` implementation, such as using the
 `SimpleDataSet` as described below, configuring things like how big the
 data set is and what the messages look like etc.  
  
 
 == DataSetSupport (abstract class)
 
-The DataSetSupport abstract class is a nice starting point for new
-DataSets, and provides some useful features to derived classes.
+The `DataSetSupport` abstract class is a nice starting point for new
+data set, and provides some useful features to derived classes.
 
 === Properties on DataSetSupport
 
@@ -117,16 +110,17 @@ DataSets, and provides some useful features to derived classes.
 |===
 |Property |Type |Default |Description
 
-|`defaultHeaders` |`Map<String,Object>` |`null` |Specifies the default message body. For SimpleDataSet it is a constant
-payload; though if you want to create custom payloads per message,
+|`defaultHeaders` |`Map<String,Object>` |`null` | Specify the default message body.
+For `SimpleDataSet` it is a constant payload; though if you want to create custom payloads per message,
 create your own derivation of `DataSetSupport`.
 
 |`outputTransformer` |`org.apache.camel.Processor` |null |
 
-|`size` |`long` |`10` |Specifies how many messages to send/consume.
+|`size` |`long` |`10` | Specify how many messages to send/consume.
 
-|`reportCount` |`long` |`-1` |Specifies the number of messages to be received before reporting
-progress. Useful for showing progress of a large load test. If < 0, then
+|`reportCount` |`long` |`-1` | Specify the number of messages to be received before reporting
+progress.
+Useful for showing the progress of a large load test. If smaller than zero (` < 0`), then
 `size` / 5, if is 0 then `size`, else set to `reportCount` value.
 |===
 
@@ -140,7 +134,7 @@ The `SimpleDataSet` extends `DataSetSupport`, and adds a default body.
 |===
 |Property |Type |Default |Description
 
-|`defaultBody` |`Object` |`<hello>world!</hello>` |Specifies the default message body. By default, the `SimpleDataSet`
+|`defaultBody` |`Object` |`<hello>world!</hello>` |Specify the default message body. By default, the `SimpleDataSet`
 produces the same constant payload for each exchange. If you want to
 customize the payload for each exchange, create a Camel `Processor` and
 configure the `SimpleDataSet` to use it by setting the
@@ -158,18 +152,19 @@ bodies.
 |===
 |Property |Type |Default |Description
 
-|`defaultBodies` |`List<Object>` |`empty LinkedList<Object>` |Specifies the default message body. By default, the `ListDataSet`
+|`defaultBodies` |`List<Object>` |`empty LinkedList<Object>` |Specify the default message body. By default, the `ListDataSet`
 selects a constant payload from the list of `defaultBodies` using the
 `CamelDataSetIndex`. If you want to customize the payload, create a
 Camel `Processor` and configure the `ListDataSet` to use it by setting
 the `outputTransformer` property.
 
-|`size` |`long` |the size of the defaultBodies list |Specifies how many messages to send/consume. This value can be different
-from the size of the `defaultBodies` list. If the value is less than the
-size of the `defaultBodies` list, some of the list elements will not be
-used. If the value is greater than the size of the `defaultBodies` list,
+|`size` |`long` |the size of the defaultBodies list |Specify how many messages to send/consume.
+This value can be different from the size of the `defaultBodies` list.
+If the value is less than the size of the `defaultBodies` list, some of the list elements will not be
+used.
+If the value is greater than the size of the `defaultBodies` list,
 the payload for the exchange will be selected using the modulus of the
-`CamelDataSetIndex` and the size of the `defaultBodies` list (i.e.
+`CamelDataSetIndex` and the size of the `defaultBodies` list (i.e.,
 `CamelDataSetIndex % defaultBodies.size()` )
 |===
 
@@ -184,9 +179,9 @@ the bodies from a file.
 |===
 |Property |Type |Default |Description
 
-|`sourceFile` |`File` |null |Specifies the source file for payloads
+|`sourceFile` |`File` |null |Specify the source file for payloads
 
-|`delimiter` |`String` |\z |Specifies the delimiter pattern used by a `java.util.Scanner` to split
+|`delimiter` |`String` |\z |Specify the delimiter pattern used by a `java.util.Scanner` to split
 the file into multiple payloads.
 |===