You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by David Novogrodsky <da...@gmail.com> on 2014/04/12 23:13:42 UTC
changing sample Storm project to run on a cluster but got a time out
All,
I am changing the Storm project noted on the page -> http://www.javaworld.com/article/2078672/open-source-tools/open-source-java-projects-storm.html?page=2 to run on a local cluster on my machine.
I might have a problem with this command.
davidnovogrodsky_wrk@davidhomecomputer~/Documents/storm-simple/target$ storm jar storm-simple-1.0-SNAPSHOT.jar net.novogrodsky.PrimeNumberTopology PrimeNumberTopology -c nimbus.host=local
but I am not sure.
The topology files used to look like this:
***
public class PrimeNumberTopology
{
public static void main(String[] args)
{
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout( "spout", new NumberSpout() );
builder.setBolt( "prime", new PrimeNumberBolt() )
.shuffleGrouping("spout");
Config conf = new Config();
LocalCluster cluster = new LocalCluster();
cluster.submitTopology("test", conf, builder.createTopology());
Utils.sleep(10000);
cluster.killTopology("test");
cluster.shutdown();
}
}
***
I changed it to look like this:
***
public class PrimeNumberTopology {
public static void main(String[] args) throws Exception{
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("spout", new NumberSpout());
builder.setBolt("prime", new PrimeNumberBolt())
.shuffleGrouping("spout");
Config conf = new Config();
if (args != null && args.length > 0) {
conf.setNumWorkers(3);
StormSubmitter.submitTopology(args[0], conf, builder.createTopology());
}
else {
conf.setMaxTaskParallelism(3);
LocalCluster cluster = new LocalCluster();
cluster.submitTopology("test", conf, builder.createTopology());
Thread.sleep(10000);
cluster.shutdown();
}
}
}
***
But I get this error when I deploy it to my local cluster:
davidnovogrodsky_wrk@davidhomecomputer~/Documents/storm-simple/target$ storm jar storm-simple-1.0-SNAPSHOT.jar net.novogrodsky.PrimeNumberTopology PrimeNumberTopology -c nimbus.host=local
…
Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Operation timed out
at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:38)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:87)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:58)
at net.novogrodsky.PrimeNumberTopology.main(PrimeNumberTopology.java:42)
Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Operation timed out
at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:83)
at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:63)
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:47)
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:43)
at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36)
... 3 more
David Novogrodsky
david.novogrodsky@gmail.com
http://www.linkedin.com/in/davidnovogrodsky
Re: changing sample Storm project to run on a cluster but got a time out
Posted by Jason Jackson <ja...@gmail.com>.
There's many examples on how to use LocalCluster in
incubator-storm/examples
E.g.
LocalCluster cluster = new LocalCluster();
cluster.submitTopology("word-count", conf, builder.createTopology());
On Sat, Apr 12, 2014 at 2:13 PM, David Novogrodsky <
david.novogrodsky@gmail.com> wrote:
> All,
>
> I am changing the Storm project noted on the page ->
> http://www.javaworld.com/article/2078672/open-source-tools/open-source-java-projects-storm.html?page=2to run on a local cluster on my machine.
>
> I might have a problem with this command.
> davidnovogrodsky_wrk@davidhomecomputer~/Documents/storm-simple/target$
> storm jar storm-simple-1.0-SNAPSHOT.jar net.novogrodsky.PrimeNumberTopology
> PrimeNumberTopology -c nimbus.host=local
>
> but I am not sure.
>
> The topology files used to look like this:
> ***
> public class PrimeNumberTopology
> {
> public static void main(String[] args)
> {
> TopologyBuilder builder = new TopologyBuilder();
>
> builder.setSpout( "spout", new NumberSpout() );
> builder.setBolt( "prime", new PrimeNumberBolt() )
> .shuffleGrouping("spout");
>
>
> Config conf = new Config();
>
> LocalCluster cluster = new LocalCluster();
> cluster.submitTopology("test", conf, builder.createTopology());
> Utils.sleep(10000);
> cluster.killTopology("test");
> cluster.shutdown();
> }
> }
> ***
>
> I changed it to look like this:
> ***
> public class PrimeNumberTopology {
> public static void main(String[] args) throws Exception{
> TopologyBuilder builder = new TopologyBuilder();
>
> builder.setSpout("spout", new NumberSpout());
> builder.setBolt("prime", new PrimeNumberBolt())
> .shuffleGrouping("spout");
>
>
> Config conf = new Config();
>
> if (args != null && args.length > 0) {
> conf.setNumWorkers(3);
>
> StormSubmitter.submitTopology(args[0], conf,
> builder.createTopology());
> }
> else {
> conf.setMaxTaskParallelism(3);
>
> LocalCluster cluster = new LocalCluster();
> cluster.submitTopology("test", conf, builder.createTopology());
>
> Thread.sleep(10000);
>
> cluster.shutdown();
> }
> }
> }
> ***
>
> But I get this error when I deploy it to my local cluster:
> davidnovogrodsky_wrk@davidhomecomputer~/Documents/storm-simple/target$
> storm jar storm-simple-1.0-SNAPSHOT.jar net.novogrodsky.PrimeNumberTopology
> PrimeNumberTopology -c nimbus.host=local
> ...
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.thrift7.transport.TTransportException:
> java.net.ConnectException: Operation timed out
> at
> backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:38)
> at
> backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:87)
> at
> backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:58)
> at
> net.novogrodsky.PrimeNumberTopology.main(PrimeNumberTopology.java:42)
> Caused by: org.apache.thrift7.transport.TTransportException:
> java.net.ConnectException: Operation timed out
> at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
> at
> org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
> at
> backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:83)
> at
> backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:63)
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:47)
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:43)
> at
> backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36)
> ... 3 more
>
>
>
> David Novogrodsky
> david.novogrodsky@gmail.com
> http://www.linkedin.com/in/davidnovogrodsky
>
>
>
>
>