You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by bo...@apache.org on 2016/01/22 23:08:36 UTC
[1/2] storm git commit: Merge branch 'STORM-1486' of
https://github.com/arunmahadevan/storm into STORM-1486
Repository: storm
Updated Branches:
refs/heads/1.x-branch 130510094 -> ed72acc63
Merge branch 'STORM-1486' of https://github.com/arunmahadevan/storm into STORM-1486
STORM-1486: Fix storm-kafa documentation
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/819917fd
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/819917fd
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/819917fd
Branch: refs/heads/1.x-branch
Commit: 819917fde32fc029d32cd736fa95bba686e1563a
Parents: 1305100
Author: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Authored: Fri Jan 22 16:06:16 2016 -0600
Committer: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Committed: Fri Jan 22 16:07:29 2016 -0600
----------------------------------------------------------------------
external/storm-kafka/README.md | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/819917fd/external/storm-kafka/README.md
----------------------------------------------------------------------
diff --git a/external/storm-kafka/README.md b/external/storm-kafka/README.md
index 3a86cf0..5a34b55 100644
--- a/external/storm-kafka/README.md
+++ b/external/storm-kafka/README.md
@@ -93,26 +93,33 @@ The KafkaConfig class also has bunch of public variables that controls your appl
Most of them are self explanatory except MultiScheme.
###MultiScheme
-MultiScheme is an interface that dictates how the byte[] consumed from Kafka gets transformed into a storm tuple. It
+MultiScheme is an interface that dictates how the ByteBuffer consumed from Kafka gets transformed into a storm tuple. It
also controls the naming of your output field.
```java
- public Iterable<List<Object>> deserialize(byte[] ser);
+ public Iterable<List<Object>> deserialize(ByteBuffer ser);
public Fields getOutputFields();
```
-The default `RawMultiScheme` just takes the `byte[]` and returns a tuple with `byte[]` as is. The name of the outputField is "bytes". There are alternative implementations like `SchemeAsMultiScheme` and `KeyValueSchemeAsMultiScheme` which can convert the `byte[]` to `String`.
+The default `RawMultiScheme` just takes the `ByteBuffer` and returns a tuple with the ByteBuffer converted to a `byte[]`. The name of the outputField is "bytes". There are alternative implementations like `SchemeAsMultiScheme` and `KeyValueSchemeAsMultiScheme` which can convert the `ByteBuffer` to `String`.
There is also an extension of `SchemeAsMultiScheme`, `MessageMetadataSchemeAsMultiScheme`,
-which has an additional deserialize method that accepts the message `byte[]` in addition to the `Partition` and `offset` associated with the message.
+which has an additional deserialize method that accepts the message `ByteBuffer` in addition to the `Partition` and `offset` associated with the message.
```java
-public Iterable<List<Object>> deserializeMessageWithMetadata(byte[] message, Partition partition, long offset)
+public Iterable<List<Object>> deserializeMessageWithMetadata(ByteBuffer message, Partition partition, long offset)
```
This is useful for auditing/replaying messages from arbitrary points on a Kafka topic, saving the partition and offset of each message of a discrete stream instead of persisting the entire message.
+#### Version incompatibility
+In Storm versions prior to 1.0, the MultiScheme methods accepted a `byte[]` instead of `ByteBuffer`. The `MultScheme` and the related
+Scheme apis were changed in version 1.0 to accept a ByteBuffer instead of a byte[].
+
+This means that pre 1.0 kafka spouts will not work with Storm versions 1.0 and higher. While running topologies in Storm version 1.0
+and higher, it must be ensured that the storm-kafka version is at least 1.0. Pre 1.0 shaded topology jars that bundles
+storm-kafka classes must be rebuilt with storm-kafka version 1.0 for running in clusters with storm 1.0 and higher.
### Examples
[2/2] storm git commit: Added STORM-1486 to Changelog
Posted by bo...@apache.org.
Added STORM-1486 to Changelog
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/ed72acc6
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/ed72acc6
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/ed72acc6
Branch: refs/heads/1.x-branch
Commit: ed72acc63b5d4138a5a411b602d235376017c3b6
Parents: 819917f
Author: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Authored: Fri Jan 22 16:06:47 2016 -0600
Committer: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Committed: Fri Jan 22 16:07:41 2016 -0600
----------------------------------------------------------------------
CHANGELOG.md | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/ed72acc6/CHANGELOG.md
----------------------------------------------------------------------
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 0ef5f44..56be926 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,4 +1,5 @@
## 1.0.0
+ * STORM-1486: Fix storm-kafa documentation
* STORM-1214: add javadoc for Trident Streams and Operations
* STORM-1450: Fix minor bugs and refactor code in ResourceAwareScheduler
* STORM-1452: Fixes profiling/debugging out of the box