You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by or...@apache.org on 2025/01/23 12:24:41 UTC

(camel) branch main updated: (chores) camel-kafka: add details about improving Kafka producer performance

This is an automated email from the ASF dual-hosted git repository.

orpiske pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git


The following commit(s) were added to refs/heads/main by this push:
     new 6993f7dc113 (chores) camel-kafka: add details about improving Kafka producer performance
6993f7dc113 is described below

commit 6993f7dc1135454af6e570da6892f61cad9e0b7d
Author: Otavio Rodolfo Piske <an...@gmail.com>
AuthorDate: Thu Jan 23 11:02:13 2025 +0100

    (chores) camel-kafka: add details about improving Kafka producer performance
---
 .../camel-kafka/src/main/docs/kafka-component.adoc    | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/components/camel-kafka/src/main/docs/kafka-component.adoc b/components/camel-kafka/src/main/docs/kafka-component.adoc
index 4c738483ca8..2545744e04d 100644
--- a/components/camel-kafka/src/main/docs/kafka-component.adoc
+++ b/components/camel-kafka/src/main/docs/kafka-component.adoc
@@ -611,6 +611,25 @@ from("kafka:topic?headerDeserializer=#class:org.apache.camel.component.kafka.con
     .to("...");
 ----
 
+=== Producer Performance
+
+If the producer is performing too slowly for your needs, you may want to aggregate the exchanges before sending.
+
+[source,java]
+.Route snippet
+----
+from("source")
+    // .other route stuff
+    .aggregate(constant(true), new GroupedExchangeAggregationStrategy())
+    .to("kafka:topic");
+----
+
+The reason for this is related to how the producer handles the two different cases:
+
+* with the `aggregrate` it should process the messages in a "batch-sized chunk" semi-asynchronously (that is, send all messages in the batch and then wait for their acknowledgements)
+* without that, it sends synchronously, eventually blocking on the record metadata fetch per exchange.
+
+NOTE: the downside of this approach is an increased number of in-flight exchanges and the potential risks (even though small and rare) associated with that.
 
 == Examples