You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Pete Carlson <pe...@tetraconcepts.com> on 2013/11/21 19:13:31 UTC
Get errors when trying to run the storm-jms example topology
When I try to run the storm-jms example topology outlined
here<https://github.com/ptgoetz/storm-jms/wiki/Example%20Topology> I
get ConnectExceptions stack traces with a Connection Refused.
Am I missing a step somewhere?
Here are the steps I've taken so far:
1) Cloned the storm-jms repo from github i.e., git clone git@github.com:
ptgoetz/storm-jms.git
2) Modified the pom.xml under the storm-jms directory per this
commit<https://github.com/pcodding/storm-jms/commit/34e6ea79cce5f91b8347ffdadda02da8f7a9e05c>
to
resolve errors with mvn clean install
3) Ran mvn clean install under storm-jms directory.
4) Modified the pom.xml under the examples directory to resolve errors with
mvn clean install (changes shown below):
Modified
<dependency>
<groupId>com.github.ptgoetz</groupId>
<artifactId>storm-jms</artifactId>
<version>0.8.1-SNAPSHOT</version>
</dependency>
to
<dependency>
<groupId>com.github.ptgoetz</groupId>
<artifactId>storm-jms</artifactId>
<version>0.8.*2*-SNAPSHOT</version>
</dependency>
and changed storm.version to 0.8.2 i.e.,
<storm.version>0.8.2</storm.version>
5) Ran mvn clean install under examples directory
6) Downloaded and installed apache-activemq-5.4.0-bin.tar.gz since
storm-jms example said it uses Apache MQ 5.4.0 for JMS. I also tried
modifying the pom to use the latest apache-activemq-5.9.0, but that failed
since maven couldn't find that jar in the clojars repo.
7) Started ActiveMQ:
[accumulo@localhost apache-activemq-5.4.0]$ cd bin/
[accumulo@localhost bin]$ ./activemq
8) Ran mvn exec:java under examples directory:
[accumulo@localhost ~]$ cd storm-jms/examples/
[accumulo@localhost examples]$ mvn exec:java
After spitting out a ton of info startup messages it seemed like it may be
working, because I saw messages about opening the JMS_QUEUE_SPOUT, and the
JMS_TOPIC_SPOUT, and connecting JMS:
INFO (clojure.tools.logging$eval1$fn__7:0) - Emitting: JMS_QUEUE_SPOUT
__system ["startup"]
INFO (clojure.tools.logging$eval1$fn__7:0) - Loaded executor tasks
JMS_QUEUE_SPOUT:[9 9]
INFO (clojure.tools.logging$eval1$fn__7:0) - Opening spout
JMS_QUEUE_SPOUT:(8)
WARN (backtype.storm.contrib.jms.spout.JmsSpout:164) - *** WARNING *** :
Recovery period (-1 ms.) is less then the configured
'topology.message.timeout.secs' of 30 secs. This could lead to a message
replay flood!
DEBUG (backtype.storm.contrib.jms.bolt.JmsBolt:187) - Connecting JMS..
INFO (clojure.tools.logging$eval1$fn__7:0) - Loading executor
JMS_TOPIC_SPOUT:[14 14]
INFO (clojure.tools.logging$eval1$fn__7:0) - Emitting: JMS_TOPIC_SPOUT
__system ["startup"]
INFO (clojure.tools.logging$eval1$fn__7:0) - Loaded executor tasks
JMS_TOPIC_SPOUT:[14 14]
INFO (clojure.tools.logging$eval1$fn__7:0) - Finished loading executor
JMS_TOPIC_SPOUT:[14 14]
INFO (clojure.tools.logging$eval1$fn__7:0) - Opening spout
JMS_TOPIC_SPOUT:(14)
WARN (backtype.storm.contrib.jms.spout.JmsSpout:164) - *** WARNING *** :
Recovery period (-1 ms.) is less then the configured
'topology.message.timeout.secs' of 30 secs. This could lead to a message
replay flood!
But then I got this stack trace:
INFO (clojure.tools.logging$eval1$fn__7:0) - Worker
41f47a5c-9ece-42b7-a9d6-bbeb2a1eb756 for storm
storm-jms-example-1-1385054209 on a4b641bf-35dd-4a86-8f57-3c6ba261f3bc:1
has finished loading
WARN (backtype.storm.contrib.jms.spout.JmsSpout:187) - Error creating JMS
connection.
javax.jms.JMSException: Could not connect to broker URL:
tcp://localhost:61616. Reason: java.net.ConnectException: Connection refused
at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:35)
at
org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:286)
at
org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:230)
at
org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:178)
at backtype.storm.contrib.jms.spout.JmsSpout.open(JmsSpout.java:175)
at backtype.storm.daemon.executor$fn__3985$fn__3997.invoke(executor.clj:460)
at backtype.storm.util$async_loop$fn__465.invoke(util.clj:375)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at
org.apache.activemq.transport.tcp.TcpTransport.connect(TcpTransport.java:483)
at
org.apache.activemq.transport.tcp.TcpTransport.doStart(TcpTransport.java:446)
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:53)
at
org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
at
org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
at
org.apache.activemq.transport.WireFormatNegotiator.start(WireFormatNegotiator.java:72)
at
org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
at
org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
at
org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:266)
... 7 more
Do I need to somehow configure the URL tcp://localhost:61616 so the
ActiveMQ broker can reach it?
Thanks for your help,
Pete
--
Pete Carlson
Software Developer
Tetra Concepts LLC
Re: Get errors when trying to run the storm-jms example topology
Posted by Pete Carlson <pe...@tetraconcepts.com>.
Hi Taylor,
Thanks for the info. When I ran the netstat command it turned out that my
machine didn't have ActiveMQ up and running after all. My mistake was that
I had only been typing *./activemq* at command line instead of *./activemq
start*
With that change the storm-jms example ran fine with both ActiveMQ 5.4.0
and ActiveMQ 5.9.0 so I deleted ActiveMQ 5.4.0 from my machine.
Btw, I'm playing with the storm-jms spout because I want to use it for my
current project. I currently have Java code that generates random data as
JSON for our application. Using Spring and Camel my code sends the JSON as
JMS messages via the ActiveMQ component defined in my applicationContext.xml
The next step is that I need to integrate this random data generation with
a Storm topology. Since my spout needs to be reliable, it seems like its a
no-brainer to simply pass my JMS messages to the storm-jms spout.
Do you have any example code of how I would embed a storm-jms spout as an
instance in my applicationContext.xml? I was initially thinking I could
just use my JmsSpout class, but I have unresolved dependencies with
backtype.storm.contrib.
Our Storm topology is still being defined, so initially I just want to pass
the data to a spout and then consume it by a dummy bolt. And then later
I'll integrate with our full Storm topology.
I'm still new to Storm, so any suggestions are appreciated.
Regards,
Pete
On Thu, Nov 21, 2013 at 2:32 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
> Hi Pete,
>
> I’ve updated the version information in the poms, so users won’t have to
> go through that process anymore.
>
> I tried to reproduce the issue you are seeing but running with a fresh
> download/install of ActiveMQ 5.9.0 (no changes to the poms were necessary).
>
> The only way I could reproduce the “Connection refused” error was to run
> the example without ActiveMQ running.
>
> Are you sure you have activemq up and running on that machine? You can
> test by running the following command:
>
> netstat -an | grep 61616
>
> - Taylor
>
> On Nov 21, 2013, at 1:13 PM, Pete Carlson <pe...@tetraconcepts.com> wrote:
>
> When I try to run the storm-jms example topology outlined here<https://github.com/ptgoetz/storm-jms/wiki/Example%20Topology> I
> get ConnectExceptions stack traces with a Connection Refused.
>
> Am I missing a step somewhere?
>
> Here are the steps I've taken so far:
>
> 1) Cloned the storm-jms repo from github i.e., git clone git@github.com:
> ptgoetz/storm-jms.git
>
> 2) Modified the pom.xml under the storm-jms directory per this commit<https://github.com/pcodding/storm-jms/commit/34e6ea79cce5f91b8347ffdadda02da8f7a9e05c> to
> resolve errors with mvn clean install
>
> 3) Ran mvn clean install under storm-jms directory.
>
> 4) Modified the pom.xml under the examples directory to resolve errors
> with mvn clean install (changes shown below):
>
> Modified
>
> <dependency>
> <groupId>com.github.ptgoetz</groupId>
> <artifactId>storm-jms</artifactId>
> <version>0.8.1-SNAPSHOT</version>
> </dependency>
>
> to
>
> <dependency>
> <groupId>com.github.ptgoetz</groupId>
> <artifactId>storm-jms</artifactId>
> <version>0.8.*2*-SNAPSHOT</version>
> </dependency>
>
> and changed storm.version to 0.8.2 i.e.,
>
> <storm.version>0.8.2</storm.version>
>
> 5) Ran mvn clean install under examples directory
>
> 6) Downloaded and installed apache-activemq-5.4.0-bin.tar.gz since
> storm-jms example said it uses Apache MQ 5.4.0 for JMS. I also tried
> modifying the pom to use the latest apache-activemq-5.9.0, but that failed
> since maven couldn't find that jar in the clojars repo.
>
> 7) Started ActiveMQ:
>
> [accumulo@localhost apache-activemq-5.4.0]$ cd bin/
> [accumulo@localhost bin]$ ./activemq
>
> 8) Ran mvn exec:java under examples directory:
>
> [accumulo@localhost ~]$ cd storm-jms/examples/
> [accumulo@localhost examples]$ mvn exec:java
>
> After spitting out a ton of info startup messages it seemed like it may be
> working, because I saw messages about opening the JMS_QUEUE_SPOUT, and the
> JMS_TOPIC_SPOUT, and connecting JMS:
>
> INFO (clojure.tools.logging$eval1$fn__7:0) - Emitting: JMS_QUEUE_SPOUT
> __system ["startup"]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Loaded executor tasks
> JMS_QUEUE_SPOUT:[9 9]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Opening spout
> JMS_QUEUE_SPOUT:(8)
> WARN (backtype.storm.contrib.jms.spout.JmsSpout:164) - *** WARNING *** :
> Recovery period (-1 ms.) is less then the configured
> 'topology.message.timeout.secs' of 30 secs. This could lead to a message
> replay flood!
>
> DEBUG (backtype.storm.contrib.jms.bolt.JmsBolt:187) - Connecting JMS..
> INFO (clojure.tools.logging$eval1$fn__7:0) - Loading executor
> JMS_TOPIC_SPOUT:[14 14]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Emitting: JMS_TOPIC_SPOUT
> __system ["startup"]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Loaded executor tasks
> JMS_TOPIC_SPOUT:[14 14]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Finished loading executor
> JMS_TOPIC_SPOUT:[14 14]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Opening spout
> JMS_TOPIC_SPOUT:(14)
> WARN (backtype.storm.contrib.jms.spout.JmsSpout:164) - *** WARNING *** :
> Recovery period (-1 ms.) is less then the configured
> 'topology.message.timeout.secs' of 30 secs. This could lead to a message
> replay flood!
>
> But then I got this stack trace:
>
> INFO (clojure.tools.logging$eval1$fn__7:0) - Worker
> 41f47a5c-9ece-42b7-a9d6-bbeb2a1eb756 for storm
> storm-jms-example-1-1385054209 on a4b641bf-35dd-4a86-8f57-3c6ba261f3bc:1
> has finished loading
> WARN (backtype.storm.contrib.jms.spout.JmsSpout:187) - Error creating JMS
> connection.
> javax.jms.JMSException: Could not connect to broker URL:
> tcp://localhost:61616. Reason: java.net.ConnectException: Connection
> refused
> at
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:35)
> at
> org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:286)
> at
> org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:230)
> at
> org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:178)
> at backtype.storm.contrib.jms.spout.JmsSpout.open(JmsSpout.java:175)
> at
> backtype.storm.daemon.executor$fn__3985$fn__3997.invoke(executor.clj:460)
> at backtype.storm.util$async_loop$fn__465.invoke(util.clj:375)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at
> org.apache.activemq.transport.tcp.TcpTransport.connect(TcpTransport.java:483)
> at
> org.apache.activemq.transport.tcp.TcpTransport.doStart(TcpTransport.java:446)
> at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:53)
> at
> org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
> at
> org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
> at
> org.apache.activemq.transport.WireFormatNegotiator.start(WireFormatNegotiator.java:72)
> at
> org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
> at
> org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
> at
> org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:266)
> ... 7 more
>
>
> Do I need to somehow configure the URL tcp://localhost:61616 so the
> ActiveMQ broker can reach it?
>
> Thanks for your help,
>
> Pete
>
> --
> Pete Carlson
> Software Developer
> Tetra Concepts LLC
>
>
>
--
Pete Carlson
Software Developer
Tetra Concepts LLC
Cell: 301-526-0681
Re: Get errors when trying to run the storm-jms example topology
Posted by "P. Taylor Goetz" <pt...@gmail.com>.
Hi Pete,
I’ve updated the version information in the poms, so users won’t have to go through that process anymore.
I tried to reproduce the issue you are seeing but running with a fresh download/install of ActiveMQ 5.9.0 (no changes to the poms were necessary).
The only way I could reproduce the “Connection refused” error was to run the example without ActiveMQ running.
Are you sure you have activemq up and running on that machine? You can test by running the following command:
netstat -an | grep 61616
- Taylor
On Nov 21, 2013, at 1:13 PM, Pete Carlson <pe...@tetraconcepts.com> wrote:
> When I try to run the storm-jms example topology outlined here I get ConnectExceptions stack traces with a Connection Refused.
>
> Am I missing a step somewhere?
>
> Here are the steps I've taken so far:
>
> 1) Cloned the storm-jms repo from github i.e., git clone git@github.com:ptgoetz/storm-jms.git
>
> 2) Modified the pom.xml under the storm-jms directory per this commit to resolve errors with mvn clean install
>
> 3) Ran mvn clean install under storm-jms directory.
>
> 4) Modified the pom.xml under the examples directory to resolve errors with mvn clean install (changes shown below):
>
> Modified
>
> <dependency>
> <groupId>com.github.ptgoetz</groupId>
> <artifactId>storm-jms</artifactId>
> <version>0.8.1-SNAPSHOT</version>
> </dependency>
>
> to
>
> <dependency>
> <groupId>com.github.ptgoetz</groupId>
> <artifactId>storm-jms</artifactId>
> <version>0.8.2-SNAPSHOT</version>
> </dependency>
>
> and changed storm.version to 0.8.2 i.e.,
>
> <storm.version>0.8.2</storm.version>
>
> 5) Ran mvn clean install under examples directory
>
> 6) Downloaded and installed apache-activemq-5.4.0-bin.tar.gz since storm-jms example said it uses Apache MQ 5.4.0 for JMS. I also tried modifying the pom to use the latest apache-activemq-5.9.0, but that failed since maven couldn't find that jar in the clojars repo.
>
> 7) Started ActiveMQ:
>
> [accumulo@localhost apache-activemq-5.4.0]$ cd bin/
> [accumulo@localhost bin]$ ./activemq
>
> 8) Ran mvn exec:java under examples directory:
>
> [accumulo@localhost ~]$ cd storm-jms/examples/
> [accumulo@localhost examples]$ mvn exec:java
>
> After spitting out a ton of info startup messages it seemed like it may be working, because I saw messages about opening the JMS_QUEUE_SPOUT, and the JMS_TOPIC_SPOUT, and connecting JMS:
>
> INFO (clojure.tools.logging$eval1$fn__7:0) - Emitting: JMS_QUEUE_SPOUT __system ["startup"]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Loaded executor tasks JMS_QUEUE_SPOUT:[9 9]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Opening spout JMS_QUEUE_SPOUT:(8)
> WARN (backtype.storm.contrib.jms.spout.JmsSpout:164) - *** WARNING *** : Recovery period (-1 ms.) is less then the configured 'topology.message.timeout.secs' of 30 secs. This could lead to a message replay flood!
>
> DEBUG (backtype.storm.contrib.jms.bolt.JmsBolt:187) - Connecting JMS..
> INFO (clojure.tools.logging$eval1$fn__7:0) - Loading executor JMS_TOPIC_SPOUT:[14 14]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Emitting: JMS_TOPIC_SPOUT __system ["startup"]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Loaded executor tasks JMS_TOPIC_SPOUT:[14 14]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Finished loading executor JMS_TOPIC_SPOUT:[14 14]
> INFO (clojure.tools.logging$eval1$fn__7:0) - Opening spout JMS_TOPIC_SPOUT:(14)
> WARN (backtype.storm.contrib.jms.spout.JmsSpout:164) - *** WARNING *** : Recovery period (-1 ms.) is less then the configured 'topology.message.timeout.secs' of 30 secs. This could lead to a message replay flood!
>
> But then I got this stack trace:
>
> INFO (clojure.tools.logging$eval1$fn__7:0) - Worker 41f47a5c-9ece-42b7-a9d6-bbeb2a1eb756 for storm storm-jms-example-1-1385054209 on a4b641bf-35dd-4a86-8f57-3c6ba261f3bc:1 has finished loading
> WARN (backtype.storm.contrib.jms.spout.JmsSpout:187) - Error creating JMS connection.
> javax.jms.JMSException: Could not connect to broker URL: tcp://localhost:61616. Reason: java.net.ConnectException: Connection refused
> at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:35)
> at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:286)
> at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:230)
> at org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:178)
> at backtype.storm.contrib.jms.spout.JmsSpout.open(JmsSpout.java:175)
> at backtype.storm.daemon.executor$fn__3985$fn__3997.invoke(executor.clj:460)
> at backtype.storm.util$async_loop$fn__465.invoke(util.clj:375)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at org.apache.activemq.transport.tcp.TcpTransport.connect(TcpTransport.java:483)
> at org.apache.activemq.transport.tcp.TcpTransport.doStart(TcpTransport.java:446)
> at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:53)
> at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
> at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
> at org.apache.activemq.transport.WireFormatNegotiator.start(WireFormatNegotiator.java:72)
> at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
> at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:58)
> at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:266)
> ... 7 more
>
>
> Do I need to somehow configure the URL tcp://localhost:61616 so the ActiveMQ broker can reach it?
>
> Thanks for your help,
>
> Pete
>
> --
> Pete Carlson
> Software Developer
> Tetra Concepts LLC
>