You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@karaf.apache.org by jb...@apache.org on 2016/03/23 16:26:58 UTC
karaf-decanter git commit: KARAF-4438 - Expose the max.request.size
in kafka appender and increase default value to deal with large messages
Repository: karaf-decanter
Updated Branches:
refs/heads/master 1604d5c63 -> e5ab9d0c8
KARAF-4438 - Expose the max.request.size in kafka appender and increase default value to deal with large messages
Project: http://git-wip-us.apache.org/repos/asf/karaf-decanter/repo
Commit: http://git-wip-us.apache.org/repos/asf/karaf-decanter/commit/e5ab9d0c
Tree: http://git-wip-us.apache.org/repos/asf/karaf-decanter/tree/e5ab9d0c
Diff: http://git-wip-us.apache.org/repos/asf/karaf-decanter/diff/e5ab9d0c
Branch: refs/heads/master
Commit: e5ab9d0c830eef766d8f2f1d4d2ba94ad261b7dc
Parents: 1604d5c
Author: Jean-Baptiste Onofré <jb...@apache.org>
Authored: Wed Mar 23 16:26:18 2016 +0100
Committer: Jean-Baptiste Onofré <jb...@apache.org>
Committed: Wed Mar 23 16:26:18 2016 +0100
----------------------------------------------------------------------
.../src/main/cfg/org.apache.karaf.decanter.appender.kafka.cfg | 3 +++
.../org/apache/karaf/decanter/appender/kafka/KafkaAppender.java | 3 +++
manual/src/main/asciidoc/user-guide/appenders.adoc | 5 +++++
3 files changed, 11 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/karaf-decanter/blob/e5ab9d0c/appender/kafka/src/main/cfg/org.apache.karaf.decanter.appender.kafka.cfg
----------------------------------------------------------------------
diff --git a/appender/kafka/src/main/cfg/org.apache.karaf.decanter.appender.kafka.cfg b/appender/kafka/src/main/cfg/org.apache.karaf.decanter.appender.kafka.cfg
index 5821bc0..e43c24f 100644
--- a/appender/kafka/src/main/cfg/org.apache.karaf.decanter.appender.kafka.cfg
+++ b/appender/kafka/src/main/cfg/org.apache.karaf.decanter.appender.kafka.cfg
@@ -36,6 +36,9 @@
# Producer request timeout
# request.timeout.ms=5000
+# Max size of the request
+# max.request.size=2097152
+
# Name of the topic
# topic=decanter
http://git-wip-us.apache.org/repos/asf/karaf-decanter/blob/e5ab9d0c/appender/kafka/src/main/java/org/apache/karaf/decanter/appender/kafka/KafkaAppender.java
----------------------------------------------------------------------
diff --git a/appender/kafka/src/main/java/org/apache/karaf/decanter/appender/kafka/KafkaAppender.java b/appender/kafka/src/main/java/org/apache/karaf/decanter/appender/kafka/KafkaAppender.java
index 6c7a31c..54e5310 100644
--- a/appender/kafka/src/main/java/org/apache/karaf/decanter/appender/kafka/KafkaAppender.java
+++ b/appender/kafka/src/main/java/org/apache/karaf/decanter/appender/kafka/KafkaAppender.java
@@ -83,6 +83,9 @@ public class KafkaAppender implements EventHandler {
String requestTimeoutMs = getValue(config, "request.timeout.ms", "5000");
properties.put("request.timeout.ms", requestTimeoutMs);
+ String maxRequestSize = getValue(config, "max.request.size", "2097152");
+ properties.put("max.request.size", maxRequestSize);
+
String valueSerializer = getValue(config, "value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
properties.put("value.serializer", valueSerializer);
http://git-wip-us.apache.org/repos/asf/karaf-decanter/blob/e5ab9d0c/manual/src/main/asciidoc/user-guide/appenders.adoc
----------------------------------------------------------------------
diff --git a/manual/src/main/asciidoc/user-guide/appenders.adoc b/manual/src/main/asciidoc/user-guide/appenders.adoc
index f43a26c..0c615f0 100644
--- a/manual/src/main/asciidoc/user-guide/appenders.adoc
+++ b/manual/src/main/asciidoc/user-guide/appenders.adoc
@@ -934,6 +934,9 @@ This feature installs a default `etc/org.apache.karaf.decanter.appender.kafka.cf
# Producer request timeout
# request.timeout.ms=5000
+# Max size of the request
+# max.request.size=2097152
+
# Name of the topic
# topic=decanter
@@ -982,6 +985,8 @@ This file allows you to define how the messages are sent to the Kafka broker:
* the `buffer.memory` defines the size of the buffer the appender uses to send to the Kafka broker. The default value is 33554432.
* the `key.serializer` defines the full qualified class name of the Serializer used to serializer the keys. The default is a String serializer (`org.apache.kafka.common.serialization.StringSerializer`).
* the `value.serializer` defines the full qualified class name of the Serializer used to serializer the values. The default is a String serializer (`org.apache.kafka.common.serialization.StringSerializer`).
+* the `request.timeout.ms` is the time the producer wait before considering the message production on the broker fails (default is 5s).
+* the `max.request.size` is the max size of the request sent to the broker (default is 2097152 bytes).
* the `topic` defines the name of the topic where to send data on the Kafka broker.
It's also possible to enable SSL security (with Kafka 0.9.x) using the SSL properties.