You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by sr...@apache.org on 2015/03/30 07:10:43 UTC
[1/4] storm git commit: Remove unused code
Repository: storm
Updated Branches:
refs/heads/master bb38f176d -> 0d5d4b129
Remove unused code
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/286c52bb
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/286c52bb
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/286c52bb
Branch: refs/heads/master
Commit: 286c52bbaea9c83d964ceacac1b44d5815294d3e
Parents: a8a0dfd
Author: Jungtaek Lim <ka...@gmail.com>
Authored: Wed Mar 25 18:30:52 2015 +0900
Committer: Jungtaek Lim <ka...@gmail.com>
Committed: Wed Mar 25 18:30:52 2015 +0900
----------------------------------------------------------------------
.../redis/topology/PersistentWordCount.java | 35 --------------------
1 file changed, 35 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/286c52bb/external/storm-redis/src/test/java/org/apache/storm/redis/topology/PersistentWordCount.java
----------------------------------------------------------------------
diff --git a/external/storm-redis/src/test/java/org/apache/storm/redis/topology/PersistentWordCount.java b/external/storm-redis/src/test/java/org/apache/storm/redis/topology/PersistentWordCount.java
index 14a969d..77c6ee8 100644
--- a/external/storm-redis/src/test/java/org/apache/storm/redis/topology/PersistentWordCount.java
+++ b/external/storm-redis/src/test/java/org/apache/storm/redis/topology/PersistentWordCount.java
@@ -45,41 +45,6 @@ public class PersistentWordCount {
private static final String TEST_REDIS_HOST = "127.0.0.1";
private static final int TEST_REDIS_PORT = 6379;
- public static class StoreCountRedisBolt extends AbstractRedisBolt {
- private static final Logger LOG = LoggerFactory.getLogger(StoreCountRedisBolt.class);
-
- public StoreCountRedisBolt(JedisPoolConfig config) {
- super(config);
- }
-
- public StoreCountRedisBolt(JedisClusterConfig config) {
- super(config);
- }
-
- @Override
- public void execute(Tuple input) {
- String word = input.getStringByField("word");
- int count = input.getIntegerByField("count");
-
- JedisCommands commands = null;
- try {
- commands = getInstance();
- commands.incrBy(word, count);
- } catch (JedisConnectionException e) {
- throw new RuntimeException("Unfortunately, this test requires redis-server running", e);
- } catch (JedisException e) {
- LOG.error("Exception occurred from Jedis/Redis", e);
- } finally {
- returnInstance(commands);
- this.collector.ack(input);
- }
- }
-
- @Override
- public void declareOutputFields(OutputFieldsDeclarer declarer) {
- }
- }
-
public static void main(String[] args) throws Exception {
Config config = new Config();
[4/4] storm git commit: Added STORM-724 to CHANGELOG.
Posted by sr...@apache.org.
Added STORM-724 to CHANGELOG.
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/0d5d4b12
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/0d5d4b12
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/0d5d4b12
Branch: refs/heads/master
Commit: 0d5d4b129e34efa6f52d8c5f5fd92ca23e3fcb6a
Parents: c8d3127
Author: Sriharsha Chintalapani <ha...@hortonworks.com>
Authored: Sun Mar 29 22:10:01 2015 -0700
Committer: Sriharsha Chintalapani <ha...@hortonworks.com>
Committed: Sun Mar 29 22:10:01 2015 -0700
----------------------------------------------------------------------
CHANGELOG.md | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/0d5d4b12/CHANGELOG.md
----------------------------------------------------------------------
diff --git a/CHANGELOG.md b/CHANGELOG.md
index af9627f..cc2b5dc 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,4 +1,5 @@
## 0.11.0
+ * STORM-724: Document RedisStoreBolt and RedisLookupBolt which is missed.
* STORM-711: All connectors should use collector.reportError and tuple anchoring.
* STORM-714: Make CSS more consistent with self, prev release
* STORM-703: With hash key option for RedisMapState, only get values for keys in batch
[2/4] storm git commit: [storm-redis] Include explanation of basic
bolts
Posted by sr...@apache.org.
[storm-redis] Include explanation of basic bolts
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/4b9b9767
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/4b9b9767
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/4b9b9767
Branch: refs/heads/master
Commit: 4b9b9767c2bcc9b10a541d950a81827d89b4607e
Parents: 286c52b
Author: Jungtaek Lim <ka...@gmail.com>
Authored: Wed Mar 25 18:31:03 2015 +0900
Committer: Jungtaek Lim <ka...@gmail.com>
Committed: Wed Mar 25 18:31:03 2015 +0900
----------------------------------------------------------------------
external/storm-redis/README.md | 108 +++++++++++++++++++++++++++++++++++-
1 file changed, 106 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/storm/blob/4b9b9767/external/storm-redis/README.md
----------------------------------------------------------------------
diff --git a/external/storm-redis/README.md b/external/storm-redis/README.md
index b01854f..32480e6 100644
--- a/external/storm-redis/README.md
+++ b/external/storm-redis/README.md
@@ -2,6 +2,8 @@
Storm/Trident integration for [Redis](http://redis.io/)
+Storm-redis uses Jedis for Redis client.
+
## Usage
### How do I use it?
@@ -12,12 +14,114 @@ use it as a maven dependency:
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-redis</artifactId>
- <version>0.10.0</version>
+ <version>{storm.version}</version>
<type>jar</type>
</dependency>
```
-### AbstractRedisBolt usage:
+### For normal Bolt
+
+Storm-redis provides basic Bolt implementations, ```RedisLookupBolt``` and ```RedisStoreBolt```.
+
+As name represents its usage, ```RedisLookupBolt``` retrieves value from Redis using key, and ```RedisStoreBolt``` stores key / value to Redis. One tuple will be matched to one key / value pair, and you can define match pattern to ```TupleMapper```.
+
+You can also choose data type from ```RedisDataTypeDescription``` to use. Please refer ```RedisDataTypeDescription.RedisDataType``` to see what data types are supported. In some data types (hash and sorted set), it requires additional key and converted key from tuple becomes element.
+
+These interfaces are combined with ```RedisLookupMapper``` and ```RedisStoreMapper``` which fit ```RedisLookupBolt``` and ```RedisStoreBolt``` respectively.
+
+#### RedisLookupBolt example
+
+```java
+
+class WordCountRedisLookupMapper implements RedisLookupMapper {
+ private RedisDataTypeDescription description;
+ private final String hashKey = "wordCount";
+
+ public WordCountRedisLookupMapper() {
+ description = new RedisDataTypeDescription(
+ RedisDataTypeDescription.RedisDataType.HASH, hashKey);
+ }
+
+ @Override
+ public List<Values> toTuple(ITuple input, Object value) {
+ String member = getKeyFromTuple(input);
+ List<Values> values = Lists.newArrayList();
+ values.add(new Values(member, value));
+ return values;
+ }
+
+ @Override
+ public void declareOutputFields(OutputFieldsDeclarer declarer) {
+ declarer.declare(new Fields("wordName", "count"));
+ }
+
+ @Override
+ public RedisDataTypeDescription getDataTypeDescription() {
+ return description;
+ }
+
+ @Override
+ public String getKeyFromTuple(ITuple tuple) {
+ return tuple.getStringByField("word");
+ }
+
+ @Override
+ public String getValueFromTuple(ITuple tuple) {
+ return null;
+ }
+}
+
+```
+
+```java
+
+JedisPoolConfig poolConfig = new JedisPoolConfig.Builder()
+ .setHost(host).setPort(port).build();
+RedisLookupMapper lookupMapper = new WordCountRedisLookupMapper();
+RedisLookupBolt lookupBolt = new RedisLookupBolt(poolConfig, lookupMapper);
+```
+
+#### RedisStoreBolt example
+
+```java
+
+class WordCountStoreMapper implements RedisStoreMapper {
+ private RedisDataTypeDescription description;
+ private final String hashKey = "wordCount";
+
+ public WordCountStoreMapper() {
+ description = new RedisDataTypeDescription(
+ RedisDataTypeDescription.RedisDataType.HASH, hashKey);
+ }
+
+ @Override
+ public RedisDataTypeDescription getDataTypeDescription() {
+ return description;
+ }
+
+ @Override
+ public String getKeyFromTuple(ITuple tuple) {
+ return tuple.getStringByField("word");
+ }
+
+ @Override
+ public String getValueFromTuple(ITuple tuple) {
+ return tuple.getStringByField("count");
+ }
+}
+```
+
+```java
+
+JedisPoolConfig poolConfig = new JedisPoolConfig.Builder()
+ .setHost(host).setPort(port).build();
+RedisStoreMapper storeMapper = new WordCountStoreMapper();
+RedisStoreBolt storeBolt = new RedisStoreBolt(poolConfig, storeMapper);
+```
+
+### For non-simple Bolt
+
+If your scenario doesn't fit ```RedisStoreBolt``` and ```RedisLookupBolt```, storm-redis also provides ```AbstractRedisBolt``` to let you extend and apply your business logic.
```java
[3/4] storm git commit: Merge branch 'STORM-724' of
https://github.com/HeartSaVioR/storm into STORM-724
Posted by sr...@apache.org.
Merge branch 'STORM-724' of https://github.com/HeartSaVioR/storm into STORM-724
Project: http://git-wip-us.apache.org/repos/asf/storm/repo
Commit: http://git-wip-us.apache.org/repos/asf/storm/commit/c8d31271
Tree: http://git-wip-us.apache.org/repos/asf/storm/tree/c8d31271
Diff: http://git-wip-us.apache.org/repos/asf/storm/diff/c8d31271
Branch: refs/heads/master
Commit: c8d31271a0c8a96749499e402d3c2cfeae63de9f
Parents: bb38f17 4b9b976
Author: Sriharsha Chintalapani <ha...@hortonworks.com>
Authored: Sun Mar 29 22:01:38 2015 -0700
Committer: Sriharsha Chintalapani <ha...@hortonworks.com>
Committed: Sun Mar 29 22:01:38 2015 -0700
----------------------------------------------------------------------
external/storm-redis/README.md | 108 ++++++++++++++++++-
.../redis/topology/PersistentWordCount.java | 35 ------
2 files changed, 106 insertions(+), 37 deletions(-)
----------------------------------------------------------------------