You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by kishorvpatil <gi...@git.apache.org> on 2018/04/23 06:45:06 UTC
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
GitHub user kishorvpatil opened a pull request:
https://github.com/apache/storm/pull/2641
[STORM-3037] Lowering CheckStyle Violations across all modules
Below is the list of modules affected by this patch. The max allowed violations are going down by 17000+.
| Module Name | Max Allowed Exceptions Before | Max Allowed Exceptions After |
| ----------- | ----------------------------- | ---------------------------- |
| storm-client | 10079 | 3298 |
| storm-server | 2585 | 783 |
| storm-eventhubs | 1765 | 45 |
| storm-starter | 1538 | 263 |
| storm-hdfs | 1406 | 189 |
| storm-sql-core | 1286 | 59 |
| storm-redis | 602 | 64 |
| storm-cassandra | 578 | 159 |
| storm-kafka | 557 | 180 |
| storm-hbase | 371 | 100 |
| storm-maven-plugins | 269 | 11 |
| storm-hive | 259 | 58 |
| storm-core | 254 | 73 |
| storm-jms | 235 | 63 |
| storm-hdfs-examples | 224 | 29 |
| storm-kafka-monitor | 178 | 87 |
| storm-mqtt | 158 | 39 |
| storm-jdbc | 149 | 36 |
| storm-solr | 108 | 47 |
| storm-perf | 100 | 65 |
| storm-hbase-examples | 55 | 16 |
| maven-shade-clojure-transformer | 16 | 0 |
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kishorvpatil/incubator-storm fix-style
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/2641.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2641
----
commit 8b34c6e7d15e7f79bb9e939ae2cb4a6d5f893eb7
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T00:16:49Z
Fixing stylecheck problems with storm-client
commit 1d8a9b6243494b11f140343819e67fc84e701200
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T00:46:33Z
Fixing stylecheck problemswith storm-sql-core
commit f1c0bcbed16085699f5fa809afb7c58f9925ce68
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T00:53:42Z
Fixing stylecheck problemswith storm-server
commit fc1cf09b05c86dc5d1802049e7b650ceb52e54d4
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T01:12:29Z
Fixing stylecheck problems with storm-core
commit 18723171612bdfe818929297378433a3c069e4e7
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T01:35:57Z
Fixing stylecheck problems with storm-eventhubs
commit 7da98cf0c5d3e23fac42871974ff8017924673c5
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T01:45:33Z
Fixing stylecheck problems with storm-hdfs
commit 81ec15d1096cd526b94313661e7b5de7ed1791d0
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T02:36:19Z
Fixing stylecheck problems with storm-starter
commit e3f5b138eb33e0b67c98a3ba3d8b26d70c988537
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T02:41:44Z
Fixing stylecheck problems with storm-redis
commit 1a2d131f99e62823bf15fa958cc676a51efda10c
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T02:44:56Z
Fixing stylecheck problems with storm-cassandra
commit 4fe4f04bb9750301b96f5c20142acb9a9a6a6000
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T02:59:46Z
Fixing stylecheck problems with storm-kafka
commit 84084ab0ac16390f24e7f24a1d9d1693062cb023
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T03:03:52Z
Fixing stylecheck problems with storm-maven-plugins
commit 6ccf6a0a0954590e3db4c95a3f22b504a5a72757
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T03:05:45Z
Fixing stylecheck problems with maven-shade-clojure-transformer
commit 880d14f1e7c6d450375b195f3aa5bc4045151fab
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T03:08:31Z
Fixing stylecheck problems with storm-hbase
commit f4ba7c952825ef7b0ee10bd1861dd0f288fe7761
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T03:11:24Z
Fixing stylecheck problems with storm-hbase-examples
commit 0e409ecd8b2d6956ed38f78bb068ce8fef67e83b
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T03:14:26Z
Fixing stylecheck problems with storm-perf
commit 5fc4e9f0bdf7a58852f7c27a1f8049e2bb3776a5
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T03:19:45Z
Fixing stylecheck problems with storm-hive
commit 224633d3ccbaa843c7f94c709d5cc573b1c59845
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T03:21:46Z
Fixing stylecheck problems with storm-kafka-monitor
commit 95602b1be7493c33b7dc8c3a8cb6e406b59e907d
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T03:23:30Z
Fixing stylecheck problems with storm-jms
commit 53adebcfec5dec5e7a59f0a9108da734d3a3f01a
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T03:24:58Z
Fixing stylecheck problems with storm-solr
commit 6d20c4af585611c6d317ad817b0a0b4b172a40ce
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T03:26:15Z
Fixing stylecheck problems with storm-jdbc
commit f7f3524d1401d76e1779c823391dce46e58acf48
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T03:32:01Z
Fixing stylecheck problems with storm-mqtt
commit e8ffac458baf6cc7a90dd0ced78cc01f0ffb4229
Author: Kishor Patil <kp...@...>
Date: 2018-04-23T04:44:09Z
Fixing stylecheck problems with storm-hdfs-examples
----
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by Ethanlm <gi...@git.apache.org>.
Github user Ethanlm commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183398979
--- Diff: examples/storm-perf/src/main/java/org/apache/storm/perf/JCQueuePerfTest.java ---
@@ -28,15 +27,15 @@
// Usage: Let it and then explicitly terminate.
// Metrics will be printed when application is terminated.
public static void main(String[] args) throws Exception {
-// oneProducer1Consumer(1000); // -- measurement 1
-// twoProducer1Consumer(1000); // -- measurement 2
-// threeProducer1Consumer(1); // -- measurement 3
+ // oneProducer1Consumer(1000); // -- measurement 1
+ // twoProducer1Consumer(1000); // -- measurement 2
+ // threeProducer1Consumer(1); // -- measurement 3
--- End diff --
Do we want to just remove these? Otherwise, delete the spaces between `//` and the actual code, e.g. `//threeProducer1Consumer(1);` to make it look better
---
[GitHub] storm issue #2641: [STORM-3037] Lowering CheckStyle Violations across all mo...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the issue:
https://github.com/apache/storm/pull/2641
Still +1
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by Ethanlm <gi...@git.apache.org>.
Github user Ethanlm commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183402104
--- Diff: examples/storm-perf/src/main/java/org/apache/storm/perf/utils/BasicMetricsCollector.java ---
@@ -192,10 +191,12 @@ void writeHeader(PrintWriter writer) {
header.add(SPOUT_MAX_COMPLETE_LATENCY);
}
- writer.println("\n------------------------------------------------------------------------------------------------------------------");
+ writer.println(
+ "\n------------------------------------------------------------------------------------------------------------------");
String str = Utils.join(header, ",");
writer.println(str);
- writer.println("------------------------------------------------------------------------------------------------------------------");
+ writer
+ .println("------------------------------------------------------------------------------------------------------------------");
--- End diff --
Would it be good to be consistent about this `writer.println()`? Here we have
```
writer.println(
"\n---...
```
and
```
writer
.println("---
```
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by kishorvpatil <gi...@git.apache.org>.
Github user kishorvpatil commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r184525490
--- Diff: examples/storm-starter/src/jvm/org/apache/storm/starter/BlobStoreAPIWordCountTopology.java ---
@@ -38,176 +47,66 @@
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.topology.base.BaseBasicBolt;
import org.apache.storm.topology.base.BaseRichSpout;
-import org.apache.storm.blobstore.BlobStoreAclHandler;
import org.apache.storm.tuple.Fields;
import org.apache.storm.tuple.Tuple;
import org.apache.storm.tuple.Values;
import org.apache.storm.utils.Utils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import java.io.BufferedReader;
-import java.io.BufferedWriter;
-import java.io.File;
-import java.io.FileReader;
-import java.io.FileWriter;
-import java.io.IOException;
-import java.util.HashSet;
-import java.util.Iterator;
-import java.util.LinkedList;
-import java.util.List;
-import java.util.Map;
-import java.util.Random;
-import java.util.Set;
-import java.util.StringTokenizer;
-
public class BlobStoreAPIWordCountTopology {
+ private static final Logger LOG = LoggerFactory.getLogger(BlobStoreAPIWordCountTopology.class);
private static ClientBlobStore store; // Client API to invoke blob store API functionality
private static String key = "key";
private static String fileName = "blacklist.txt";
- private static final Logger LOG = LoggerFactory.getLogger(BlobStoreAPIWordCountTopology.class);
public static void prepare() {
Config conf = new Config();
conf.putAll(Utils.readStormConfig());
store = Utils.getClientBlobStore(conf);
}
- // Spout implementation
- public static class RandomSentenceSpout extends BaseRichSpout {
- SpoutOutputCollector _collector;
-
- @Override
- public void open(Map<String, Object> conf, TopologyContext context, SpoutOutputCollector collector) {
- _collector = collector;
- }
-
- @Override
- public void nextTuple() {
- Utils.sleep(100);
- _collector.emit(new Values(getRandomSentence()));
- }
-
- @Override
- public void ack(Object id) {
- }
-
- @Override
- public void fail(Object id) {
- }
-
- @Override
- public void declareOutputFields(OutputFieldsDeclarer declarer) {
- declarer.declare(new Fields("sentence"));
- }
-
- }
-
- // Bolt implementation
- public static class SplitSentence extends ShellBolt implements IRichBolt {
-
- public SplitSentence() {
- super("python", "splitsentence.py");
- }
-
- @Override
- public void declareOutputFields(OutputFieldsDeclarer declarer) {
- declarer.declare(new Fields("word"));
- }
-
- @Override
- public Map<String, Object> getComponentConfiguration() {
- return null;
- }
- }
-
- public static class FilterWords extends BaseBasicBolt {
- boolean poll = false;
- long pollTime;
- Set<String> wordSet;
- @Override
- public void execute(Tuple tuple, BasicOutputCollector collector) {
- String word = tuple.getString(0);
- // Thread Polling every 5 seconds to update the wordSet seconds which is
- // used in FilterWords bolt to filter the words
- try {
- if (!poll) {
- wordSet = parseFile(fileName);
- pollTime = System.currentTimeMillis();
- poll = true;
- } else {
- if ((System.currentTimeMillis() - pollTime) > 5000) {
- wordSet = parseFile(fileName);
- pollTime = System.currentTimeMillis();
- }
- }
- } catch (IOException exp) {
- throw new RuntimeException(exp);
- }
- if (wordSet !=null && !wordSet.contains(word)) {
- collector.emit(new Values(word));
- }
- }
-
- @Override
- public void declareOutputFields(OutputFieldsDeclarer declarer) {
- declarer.declare(new Fields("word"));
- }
- }
-
- public void buildAndLaunchWordCountTopology(String[] args) {
- TopologyBuilder builder = new TopologyBuilder();
- builder.setSpout("spout", new RandomSentenceSpout(), 5);
- builder.setBolt("split", new SplitSentence(), 8).shuffleGrouping("spout");
- builder.setBolt("filter", new FilterWords(), 6).shuffleGrouping("split");
-
- Config conf = new Config();
- conf.setDebug(true);
- try {
- conf.setNumWorkers(3);
- StormSubmitter.submitTopologyWithProgressBar(args[0], conf, builder.createTopology());
- } catch (InvalidTopologyException | AuthorizationException | AlreadyAliveException exp) {
- throw new RuntimeException(exp);
- }
- }
-
--- End diff --
There are checkstyle condition that decide import order, modified overloaded method order and at-clause orders. To avoid these violations, we need to put these methods in right order.
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by Ethanlm <gi...@git.apache.org>.
Github user Ethanlm commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183395986
--- Diff: examples/storm-hdfs-examples/src/main/java/org/apache/storm/hdfs/bolt/SequenceFileTopology.java ---
@@ -151,15 +145,15 @@ public void nextTuple() {
}
count++;
total++;
- if(count > 20000){
+ if (count > 20000) {
count = 0;
System.out.println("Pending count: " + this.pending.size() + ", total: " + this.total);
}
Thread.yield();
}
public void ack(Object msgId) {
-// System.out.println("ACK");
+ // System.out.println("ACK");
--- End diff --
Should we just remove this
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by kishorvpatil <gi...@git.apache.org>.
Github user kishorvpatil commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183772336
--- Diff: examples/storm-hbase-examples/src/main/java/org/apache/storm/hbase/topology/LookupWordCount.java ---
@@ -1,25 +1,19 @@
/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version
+ * 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
--- End diff --
The line length allowed is 140 characters. So the paragraphs are reformatted
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by Ethanlm <gi...@git.apache.org>.
Github user Ethanlm commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183402311
--- Diff: examples/storm-perf/src/main/java/org/apache/storm/perf/utils/MetricsSample.java ---
@@ -160,7 +159,7 @@ private static MetricsSample getMetricsSample(TopologyInfo topInfo) {
ret.spoutEmitted = spoutEmitted;
ret.spoutTransferred = spoutTransferred;
ret.sampleTime = System.currentTimeMillis();
-// ret.numSupervisors = clusterSummary.get_supervisors_size();
+ // ret.numSupervisors = clusterSummary.get_supervisors_size();
--- End diff --
remove this line ? or delete the space?
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by Ethanlm <gi...@git.apache.org>.
Github user Ethanlm commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183400894
--- Diff: examples/storm-perf/src/main/java/org/apache/storm/perf/spout/StringGenSpout.java ---
@@ -22,33 +22,40 @@
import java.util.Collections;
import java.util.List;
import java.util.Map;
-
import org.apache.commons.lang.RandomStringUtils;
import org.apache.storm.spout.SpoutOutputCollector;
import org.apache.storm.task.TopologyContext;
import org.apache.storm.topology.OutputFieldsDeclarer;
import org.apache.storm.topology.base.BaseRichSpout;
import org.apache.storm.tuple.Fields;
-/** Spout pre-computes a list with 30k fixed length random strings.
- * Emits sequentially from this list, over and over again.
+/**
+ * Spout pre-computes a list with 30k fixed length random strings. Emits sequentially from this list, over and over again.
*/
public class StringGenSpout extends BaseRichSpout {
private static final String DEFAULT_FIELD_NAME = "str";
- private int strLen;
private final int strCount = 30_000;
+ ArrayList<String> records;
+ private int strLen;
private String fieldName = DEFAULT_FIELD_NAME;
private SpoutOutputCollector collector = null;
- ArrayList<String> records;
private int curr = 0;
private int count = 0;
public StringGenSpout(int strLen) {
this.strLen = strLen;
}
+ private static ArrayList<String> genStringList(int strLen, int count) {
+ ArrayList<String> result = new ArrayList<String>(count);
+ for (int i = 0; i < count; i++) {
+ result.add(RandomStringUtils.random(strLen));
+ }
+ return result;
+ }
+
--- End diff --
It looks like you just moved the location of the function. Is there any particular reason to do it?
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by kishorvpatil <gi...@git.apache.org>.
Github user kishorvpatil commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r184525138
--- Diff: examples/storm-perf/src/main/java/org/apache/storm/perf/utils/MetricsSample.java ---
@@ -160,7 +159,7 @@ private static MetricsSample getMetricsSample(TopologyInfo topInfo) {
ret.spoutEmitted = spoutEmitted;
ret.spoutTransferred = spoutTransferred;
ret.sampleTime = System.currentTimeMillis();
-// ret.numSupervisors = clusterSummary.get_supervisors_size();
+ // ret.numSupervisors = clusterSummary.get_supervisors_size();
--- End diff --
Addressed
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by Ethanlm <gi...@git.apache.org>.
Github user Ethanlm commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183397732
--- Diff: examples/storm-perf/src/main/java/org/apache/storm/perf/FileReadWordCountTopo.java ---
@@ -1,25 +1,24 @@
/*
--- End diff --
Should be `/**` here?
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by kishorvpatil <gi...@git.apache.org>.
Github user kishorvpatil commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183771851
--- Diff: examples/storm-hdfs-examples/src/main/java/org/apache/storm/hdfs/bolt/SequenceFileTopology.java ---
@@ -151,15 +145,15 @@ public void nextTuple() {
}
count++;
total++;
- if(count > 20000){
+ if (count > 20000) {
count = 0;
System.out.println("Pending count: " + this.pending.size() + ", total: " + this.total);
}
Thread.yield();
}
public void ack(Object msgId) {
-// System.out.println("ACK");
+ // System.out.println("ACK");
--- End diff --
This is attempt to only fix format to allow for checkstyle to pass it. I did not go into each comment for its validity for deletion.
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by Ethanlm <gi...@git.apache.org>.
Github user Ethanlm commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183399672
--- Diff: examples/storm-perf/src/main/java/org/apache/storm/perf/LowThroughputTopo.java ---
@@ -64,7 +63,7 @@ static StormTopology getTopology(Map<String, Object> conf) {
BoltDeclarer bd = builder.setBolt(BOLT_ID, bolt, Helper.getInt(conf, BOLT_COUNT, 1));
bd.localOrShuffleGrouping(SPOUT_ID);
-// bd.shuffleGrouping(SPOUT_ID);
+ // bd.shuffleGrouping(SPOUT_ID);
--- End diff --
nit: space?
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/2641
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by Ethanlm <gi...@git.apache.org>.
Github user Ethanlm commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183399450
--- Diff: examples/storm-perf/src/main/java/org/apache/storm/perf/JCToolsPerfTest.java ---
@@ -19,25 +19,24 @@
package org.apache.storm.perf;
import java.util.concurrent.locks.LockSupport;
-
import org.apache.storm.utils.MutableLong;
import org.jctools.queues.MpscArrayQueue;
public class JCToolsPerfTest {
public static void main(String[] args) throws Exception {
-// oneProducer1Consumer();
-// twoProducer1Consumer();
-// threeProducer1Consumer();
-// oneProducer2Consumers();
-// producerFwdConsumer();
-
-// JCQueue spoutQ = new JCQueue("spoutQ", 1024, 100, 0);
-// JCQueue ackQ = new JCQueue("ackQ", 1024, 100, 0);
-//
-// final AckingProducer ackingProducer = new AckingProducer(spoutQ, ackQ);
-// final Acker acker = new Acker(ackQ, spoutQ);
-//
-// runAllThds(ackingProducer, acker);
+ // oneProducer1Consumer();
+ // twoProducer1Consumer();
+ // threeProducer1Consumer();
+ // oneProducer2Consumers();
+ // producerFwdConsumer();
--- End diff --
nit: spaces between `//` and code
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by Ethanlm <gi...@git.apache.org>.
Github user Ethanlm commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183402775
--- Diff: examples/storm-starter/src/jvm/org/apache/storm/starter/BlobStoreAPIWordCountTopology.java ---
@@ -38,176 +47,66 @@
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.topology.base.BaseBasicBolt;
import org.apache.storm.topology.base.BaseRichSpout;
-import org.apache.storm.blobstore.BlobStoreAclHandler;
import org.apache.storm.tuple.Fields;
import org.apache.storm.tuple.Tuple;
import org.apache.storm.tuple.Values;
import org.apache.storm.utils.Utils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import java.io.BufferedReader;
-import java.io.BufferedWriter;
-import java.io.File;
-import java.io.FileReader;
-import java.io.FileWriter;
-import java.io.IOException;
-import java.util.HashSet;
-import java.util.Iterator;
-import java.util.LinkedList;
-import java.util.List;
-import java.util.Map;
-import java.util.Random;
-import java.util.Set;
-import java.util.StringTokenizer;
-
public class BlobStoreAPIWordCountTopology {
+ private static final Logger LOG = LoggerFactory.getLogger(BlobStoreAPIWordCountTopology.class);
private static ClientBlobStore store; // Client API to invoke blob store API functionality
private static String key = "key";
private static String fileName = "blacklist.txt";
- private static final Logger LOG = LoggerFactory.getLogger(BlobStoreAPIWordCountTopology.class);
public static void prepare() {
Config conf = new Config();
conf.putAll(Utils.readStormConfig());
store = Utils.getClientBlobStore(conf);
}
- // Spout implementation
- public static class RandomSentenceSpout extends BaseRichSpout {
- SpoutOutputCollector _collector;
-
- @Override
- public void open(Map<String, Object> conf, TopologyContext context, SpoutOutputCollector collector) {
- _collector = collector;
- }
-
- @Override
- public void nextTuple() {
- Utils.sleep(100);
- _collector.emit(new Values(getRandomSentence()));
- }
-
- @Override
- public void ack(Object id) {
- }
-
- @Override
- public void fail(Object id) {
- }
-
- @Override
- public void declareOutputFields(OutputFieldsDeclarer declarer) {
- declarer.declare(new Fields("sentence"));
- }
-
- }
-
- // Bolt implementation
- public static class SplitSentence extends ShellBolt implements IRichBolt {
-
- public SplitSentence() {
- super("python", "splitsentence.py");
- }
-
- @Override
- public void declareOutputFields(OutputFieldsDeclarer declarer) {
- declarer.declare(new Fields("word"));
- }
-
- @Override
- public Map<String, Object> getComponentConfiguration() {
- return null;
- }
- }
-
- public static class FilterWords extends BaseBasicBolt {
- boolean poll = false;
- long pollTime;
- Set<String> wordSet;
- @Override
- public void execute(Tuple tuple, BasicOutputCollector collector) {
- String word = tuple.getString(0);
- // Thread Polling every 5 seconds to update the wordSet seconds which is
- // used in FilterWords bolt to filter the words
- try {
- if (!poll) {
- wordSet = parseFile(fileName);
- pollTime = System.currentTimeMillis();
- poll = true;
- } else {
- if ((System.currentTimeMillis() - pollTime) > 5000) {
- wordSet = parseFile(fileName);
- pollTime = System.currentTimeMillis();
- }
- }
- } catch (IOException exp) {
- throw new RuntimeException(exp);
- }
- if (wordSet !=null && !wordSet.contains(word)) {
- collector.emit(new Values(word));
- }
- }
-
- @Override
- public void declareOutputFields(OutputFieldsDeclarer declarer) {
- declarer.declare(new Fields("word"));
- }
- }
-
- public void buildAndLaunchWordCountTopology(String[] args) {
- TopologyBuilder builder = new TopologyBuilder();
- builder.setSpout("spout", new RandomSentenceSpout(), 5);
- builder.setBolt("split", new SplitSentence(), 8).shuffleGrouping("spout");
- builder.setBolt("filter", new FilterWords(), 6).shuffleGrouping("split");
-
- Config conf = new Config();
- conf.setDebug(true);
- try {
- conf.setNumWorkers(3);
- StormSubmitter.submitTopologyWithProgressBar(args[0], conf, builder.createTopology());
- } catch (InvalidTopologyException | AuthorizationException | AlreadyAliveException exp) {
- throw new RuntimeException(exp);
- }
- }
-
--- End diff --
Why are we moving locations of the functions
---
[GitHub] storm pull request #2641: [STORM-3037] Lowering CheckStyle Violations across...
Posted by Ethanlm <gi...@git.apache.org>.
Github user Ethanlm commented on a diff in the pull request:
https://github.com/apache/storm/pull/2641#discussion_r183396233
--- Diff: examples/storm-hbase-examples/src/main/java/org/apache/storm/hbase/topology/LookupWordCount.java ---
@@ -1,25 +1,19 @@
/**
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version
+ * 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
--- End diff --
Why do we change this
---