You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by bo...@apache.org on 2015/01/14 17:05:45 UTC
[24/44] storm git commit: update storm-kafka README.md
update storm-kafka README.md
Update storm-kafka README.md with info about retry settings.
Also update the sentence on what SpoutConfig provides over KafkaConfig.
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/77c2df28
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/77c2df28
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/77c2df28
Branch: refs/heads/master
Commit: 77c2df28517f9b9acd87724116d9b7b3573be4fc
Parents: 8c78c13
Author: Rick Kilgore <ri...@hbo.com>
Authored: Thu Sep 11 17:29:05 2014 -0700
Committer: Rick Kilgore <ri...@hbo.com>
Committed: Thu Sep 11 17:29:05 2014 -0700
----------------------------------------------------------------------
external/storm-kafka/README.md | 22 +++++++++++++++-----
.../src/jvm/storm/kafka/SpoutConfig.java | 9 ++++++--
2 files changed, 24 insertions(+), 7 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/77c2df28/external/storm-kafka/README.md
----------------------------------------------------------------------
diff --git a/external/storm-kafka/README.md b/external/storm-kafka/README.md
index 73cf334..283a950 100644
--- a/external/storm-kafka/README.md
+++ b/external/storm-kafka/README.md
@@ -51,17 +51,29 @@ The optional ClientId is used as a part of the zookeeper path where the spout's
There are 2 extensions of KafkaConfig currently in use.
-Spoutconfig is an extension of KafkaConfig that supports 2 additional fields, zkroot and id. The Zkroot will be used
-as root to store your consumer's offset. The id should uniquely identify your spout.
+Spoutconfig is an extension of KafkaConfig that supports additional fields with ZooKeeper connection info and for controlling
+behavior specific to KafkaSpout. The Zkroot will be used as root to store your consumer's offset. The id should uniquely
+identify your spout.
```java
public SpoutConfig(BrokerHosts hosts, String topic, String zkRoot, String id);
+public SpoutConfig(BrokerHosts hosts, String topic, String id);
+```
+In addition to these parameters, SpoutConfig contains the following fields that control how KafkaSpout behaves:
+```java
+ // setting for how often to save the current kafka offset to ZooKeeper
+ public long stateUpdateIntervalMs = 2000;
+
+ // Exponential back-off retry settings. These are used when retrying messages after a bolt
+ // calls OutputCollector.fail().
+ // Note: be sure to set backtype.storm.Config.MESSAGE_TIMEOUT_SECS appropriately to prevent
+ // resubmitting the message while still retrying.
+ public long retryInitialDelayMs = 0;
+ public double retryDelayMultiplier = 1.0;
+ public long retryDelayMaxMs = 60 * 1000;
```
Core KafkaSpout only accepts an instance of SpoutConfig.
TridentKafkaConfig is another extension of KafkaConfig.
-```java
-public SpoutConfig(BrokerHosts hosts, String topic, String id);
-```
TridentKafkaEmitter only accepts TridentKafkaConfig.
The KafkaConfig class also has bunch of public variables that controls your application's behavior. Here are defaults:
http://git-wip-us.apache.org/repos/asf/storm/blob/77c2df28/external/storm-kafka/src/jvm/storm/kafka/SpoutConfig.java
----------------------------------------------------------------------
diff --git a/external/storm-kafka/src/jvm/storm/kafka/SpoutConfig.java b/external/storm-kafka/src/jvm/storm/kafka/SpoutConfig.java
index fbc887f..1c40ddf 100644
--- a/external/storm-kafka/src/jvm/storm/kafka/SpoutConfig.java
+++ b/external/storm-kafka/src/jvm/storm/kafka/SpoutConfig.java
@@ -26,10 +26,15 @@ public class SpoutConfig extends KafkaConfig implements Serializable {
public Integer zkPort = null;
public String zkRoot = null;
public String id = null;
+
+ // setting for how often to save the current kafka offset to ZooKeeper
public long stateUpdateIntervalMs = 2000;
- // Exponential back-off retry settings - note: be sure to set backtype.storm.Config.MESSAGE_TIMEOUT_SECS
- // appropriately to prevent resubmitting the message while still retrying
+ // Exponential back-off retry settings. These are used when retrying messages after a bolt
+ // calls OutputCollector.fail().
+ //
+ // Note: be sure to set backtype.storm.Config.MESSAGE_TIMEOUT_SECS appropriately to prevent
+ // resubmitting the message while still retrying.
public long retryInitialDelayMs = 0;
public double retryDelayMultiplier = 1.0;
public long retryDelayMaxMs = 60 * 1000;