You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Joaquin Menchaca <jm...@gobalto.com> on 2016/10/06 18:45:28 UTC
Storm Bug? Cannot connect to cluster
I have only the nimbus.seeds configured, do I need anything else configured
to get his to work?
$ storm --config /templates/storm.yaml localconfvalue nimbus.seeds
nimbus.seeds: [ip-10-110-20-7.us-west-2.compute.internal
ip-10-110-20-146.us-west-2.compute.internal]
$ nc -vz ip-10-110-20-7.us-west-2.compute.internal 6627
Connection to ip-10-110-20-7.us-west-2.compute.internal 6627 port [tcp/*]
succeeded!
$ nc -vz ip-10-110-20-146.us-west-2.compute.internal 6627
Connection to ip-10-110-20-146.us-west-2.compute.internal 6627 port [tcp/*]
succeeded!
$ storm --config /templates/storm.yaml list
Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name=
-Dstorm.options= -Dstorm.home=/usr/share/storm
-Dstorm.log.dir=/usr/share/storm/logs
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
-Dstorm.conf.file=/templates/storm.yaml -cp
/usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/servlet-api-2.5.jar:/usr/share/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:/usr/share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/lib/storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm-1.07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.jar:/usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/storm/lib/minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.10.4.jar:/templates:/usr/share/storm/bin
backtype.storm.command.list
1283 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
2263 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as
user:
2293 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
2348 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The
baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
Exception in thread "main" java.lang.RuntimeException:
org.apache.thrift7.transport.TTransportException:
java.net.ConnectException: Connection refused
at
backtype.storm.security.auth.TBackoffConnect.retryNext(TBackoffConnect.java:59)
at
backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:51)
at
backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103)
at
backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72)
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:69)
at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75)
at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:72)
at backtype.storm.command.list$_main.invoke(list.clj:22)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at backtype.storm.command.list.main(Unknown Source)
Caused by: org.apache.thrift7.transport.TTransportException:
java.net.ConnectException: Connection refused
at org.apache.thrift7.transport.TSocket.open(TSocket.java:187)
at
org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at
backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:103)
at
backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
... 9 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:182)
... 12 more
Re: Storm Bug? Cannot connect to cluster
Posted by Joaquin Menchaca <jm...@gobalto.com>.
Alright, ignore everything I said... I did not know storm list will need to
communicate to other systems within the cluster. I was thinking that if
the nimbus were documented, then that was all that would be needed. I did
not know that I would need to communicate to other systems.
Thus, I had to setup a /etc/hosts to refer to the other systems, no idea
which ones, so put everything, zookeepers and all.
I did not find where this is documented. :'(
Sorry if I wasted anyone's time.
Re: Storm Bug? Cannot connect to cluster
Posted by Joaquin Menchaca <jm...@gobalto.com>.
With storm 1.0.2 client, storm list seems to freeze and cause a timeout
exception. Is there an alternative way to list running topologies? It
establishes a connection 10.110.20.7, but then timeout. Does it need to
query other systems?
$ storm --config /templates/storm.yaml localconfvalue nimbus.seeds
nimbus.seeds: [ip-10-110-20-7.us-west-2.compute.internal
ip-10-110-20-146.us-west-2.compute.internal]
$ nc -vz ip-10-110-20-7.us-west-2.compute.internal 6627
Connection to ip-10-110-20-7.us-west-2.compute.internal 6627 port [tcp/*]
succeeded!
$ nc -vz ip-10-110-20-146.us-west-2.compute.internal 6627
Connection to ip-10-110-20-146.us-west-2.compute.internal 6627 port [tcp/*]
succeeded!
$ storm --config /templates/storm.yaml list
# Exception in thread "main" java.lang.RuntimeException:
org.apache.storm.thrift.transport.TTransportException:
java.net.ConnectException: Connection timed out
at
org.apache.storm.security.auth.TBackoffConnect.retryNext(TBackoffConnect.java:64)
at
org.apache.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:56)
at
org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:99)
at
org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:69)
at org.apache.storm.utils.NimbusClient.<init>(NimbusClient.java:106)
at
org.apache.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:78)
at org.apache.storm.command.list$_main.invoke(list.clj:22)
at clojure.lang.AFn.applyToHelper(AFn.java:152)
at clojure.lang.AFn.applyTo(AFn.java:144)
at org.apache.storm.command.list.main(Unknown Source)
Caused by: org.apache.storm.thrift.transport.TTransportException:
java.net.ConnectException: Connection timed out
at org.apache.storm.thrift.transport.TSocket.open(TSocket.java:226)
at
org.apache.storm.thrift.transport.TFramedTransport.open(TFramedTransport.java:81)
at
org.apache.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:103)
at
org.apache.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:53)
... 8 more
Caused by: java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.storm.thrift.transport.TSocket.open(TSocket.java:221)
... 11 more
# netstat -anp | grep 6627
tcp6 0 1 172.17.0.2:46537 52.43.119.126:6627
SYN_SENT 3598/java
tcp6 0 0 172.17.0.2:46089 10.110.20.7:6627
ESTABLISHED 3598/java
On Thu, Oct 6, 2016 at 4:36 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
> I don't think it's a bug on your part. I tested it and found that if only
> nimbus.seed is set, the 'storm list' command will attempt to connect with
> localhost.
>
> -Taylor
>
> On Oct 6, 2016, at 6:16 PM, Joaquin Menchaca <jm...@gobalto.com>
> wrote:
>
> It might be a bug on my part, trying a test, office Internet is slooow, so
> have to step outside of office to do the test...
>
> - joaquin
>
> PS - Not on the topic of writing bugs... how could I file bugs myself? I
> wanted to make a PR for some doc bugs, but don't quite know how to PR at a
> particular tag...
>
> On Thu, Oct 6, 2016 at 1:43 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
>
>> This is a bug. The `storm list` command implementation doesn’t understand
>> the “nimbus.seeds” configuration setting.
>>
>> Can you try the following?
>>
>> Add a nimbus.host entry in your storm.yaml file with ONE of your nimbus
>> hostnames.
>>
>> Let me know if that works, and I’ll file a JIRA to get this fixed.
>>
>> -Taylor
>>
>>
>> On Oct 6, 2016, at 3:31 PM, Joaquin Menchaca <jm...@gobalto.com>
>> wrote:
>>
>> I don't get it, it seems to pick up the correct configuration... but then
>> ignores the nimbus.seeds and users localhost.
>>
>> # cat /templates/storm.yaml
>> storm.zookeeper.servers:
>> - "ip-10-110-20-8.us-west-2.compute.internal"
>>
>> nimbus.seeds: ["ip-10-110-20-7.us-west-2.compute.internal", "
>> ip-10-110-20-146.us-west-2.compute.internal"]
>>
>> # storm --config /templates/storm.yaml list
>> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name=
>> -Dstorm.options= -Dstorm.home=/usr/share/storm
>> -Dstorm.log.dir=/usr/share/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
>> *-Dstorm.conf.file=/templates/storm.yaml* -cp
>> /usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-
>> slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/
>> usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/
>> servlet-api-2.5.jar:/usr/share/storm/lib/log4j-over-
>> slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:
>> /usr/share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/
>> lib/storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm
>> -1.07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.
>> jar:/usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/
>> storm/lib/minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.
>> 10.4.jar:/templates:/usr/share/storm/bin backtype.storm.command.list
>> ...
>> 2264 [main] INFO b.s.thrift - Connecting to Nimbus at* localhost:6627 *as
>> user:
>> ...
>> Caused by: java.net.ConnectException: Connection refused
>> ...
>>
>> On Thu, Oct 6, 2016 at 12:15 PM, P. Taylor Goetz <pt...@gmail.com>
>> wrote:
>>
>>> For some reason it is trying to connect to localhost:
>>>
>>> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as
>>> user:
>>>
>>> Do you have a ls ~/.storm/storm.yaml file that might be getting picked
>>> up?
>>>
>>> What are the contents of the storm.yaml you are using in your commands?
>>>
>>> -Taylor
>>>
>>>
>>> On Oct 6, 2016, at 2:45 PM, Joaquin Menchaca <jm...@gobalto.com>
>>> wrote:
>>>
>>> I have only the nimbus.seeds configured, do I need anything else
>>> configured to get his to work?
>>>
>>> $ storm --config /templates/storm.yaml localconfvalue nimbus.seeds
>>> nimbus.seeds: [ip-10-110-20-7.us-west-2.compute.internal
>>> ip-10-110-20-146.us-west-2.compute.internal]
>>>
>>> $ nc -vz ip-10-110-20-7.us-west-2.compute.internal 6627
>>> Connection to ip-10-110-20-7.us-west-2.compute.internal 6627 port
>>> [tcp/*] succeeded!
>>> $ nc -vz ip-10-110-20-146.us-west-2.compute.internal 6627
>>> Connection to ip-10-110-20-146.us-west-2.compute.internal 6627 port
>>> [tcp/*] succeeded!
>>>
>>> $ storm --config /templates/storm.yaml list
>>> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name=
>>> -Dstorm.options= -Dstorm.home=/usr/share/storm
>>> -Dstorm.log.dir=/usr/share/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
>>> -Dstorm.conf.file=/templates/storm.yaml -cp
>>> /usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-
>>> slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/
>>> usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/servl
>>> et-api-2.5.jar:/usr/share/storm/lib/log4j-over-slf4j-1.
>>> 6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:/usr/
>>> share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/lib/
>>> storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm-1.
>>> 07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.jar:/
>>> usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/storm/lib/
>>> minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.10.4.
>>> jar:/templates:/usr/share/storm/bin backtype.storm.command.list
>>> 1283 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>>> 2263 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>>> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as
>>> user:
>>> 2293 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>>> 2348 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The
>>> baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
>>> Exception in thread "main" java.lang.RuntimeException:
>>> org.apache.thrift7.transport.TTransportException:
>>> java.net.ConnectException: Connection refused
>>> at backtype.storm.security.auth.TBackoffConnect.retryNext(TBack
>>> offConnect.java:59)
>>> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRe
>>> try(TBackoffConnect.java:51)
>>> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftCl
>>> ient.java:103)
>>> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClien
>>> t.java:72)
>>> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:69)
>>> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.c
>>> lj:75)
>>> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.c
>>> lj:72)
>>> at backtype.storm.command.list$_main.invoke(list.clj:22)
>>> at clojure.lang.AFn.applyToHelper(AFn.java:152)
>>> at clojure.lang.AFn.applyTo(AFn.java:144)
>>> at backtype.storm.command.list.main(Unknown Source)
>>> Caused by: org.apache.thrift7.transport.TTransportException:
>>> java.net.ConnectException: Connection refused
>>> at org.apache.thrift7.transport.TSocket.open(TSocket.java:187)
>>> at org.apache.thrift7.transport.TFramedTransport.open(TFramedTr
>>> ansport.java:81)
>>> at backtype.storm.security.auth.SimpleTransportPlugin.connect(S
>>> impleTransportPlugin.java:103)
>>> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRe
>>> try(TBackoffConnect.java:48)
>>> ... 9 more
>>> Caused by: java.net.ConnectException: Connection refused
>>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>>> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSock
>>> etImpl.java:350)
>>> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPl
>>> ainSocketImpl.java:206)
>>> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocket
>>> Impl.java:188)
>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>>> at java.net.Socket.connect(Socket.java:589)
>>> at org.apache.thrift7.transport.TSocket.open(TSocket.java:182)
>>> ... 12 more
>>>
>>>
>>>
>>
>>
>> --
>>
>> 是故勝兵先勝而後求戰，敗兵先戰而後求勝。
>>
>>
>>
>
>
> --
>
> 是故勝兵先勝而後求戰，敗兵先戰而後求勝。
>
>
--
是故勝兵先勝而後求戰，敗兵先戰而後求勝。
Re: Storm Bug? Cannot connect to cluster
Posted by Joaquin Menchaca <jm...@gobalto.com>.
I did find that I had a dirty container. Docker is frustrating, as it was
grabbing a dusty old image off the build server, rather than pull down the
latest base image. So I actually did have storm 0.10.0 container.
When took the nuclear option and blasted images, and I now have storm
1.0.2.
But now it freezes... like... wow. totally freezes.
# storm --config /templates/storm.yaml list
Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name=
-Dstorm.options= -Dstorm.home=/usr/share/storm
-Dstorm.log.dir=/usr/share/storm/logs
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
-Dstorm.conf.file=/templates/storm.yaml -cp
/usr/share/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/share/storm/lib/storm-rename-hack-1.0.2.jar:/usr/share/storm/lib/clojure-1.7.0.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/usr/share/storm/lib/servlet-api-2.5.jar:/usr/share/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:/usr/share/storm/lib/minlog-1.3.0.jar:/usr/share/storm/lib/storm-core-1.0.2.jar:/usr/share/storm/lib/kryo-3.0.3.jar:/usr/share/storm/lib/asm-5.0.3.jar:/usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/storm/lib/disruptor-3.3.2.jar:/usr/share/storm/lib/objenesis-2.1.jar:/usr/share/storm/lib/reflectasm-1.10.1.jar:/templates:/usr/share/storm/bin
org.apache.storm.command.list
Can I do anything to debug further?
On Thu, Oct 6, 2016 at 4:36 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
> I don't think it's a bug on your part. I tested it and found that if only
> nimbus.seed is set, the 'storm list' command will attempt to connect with
> localhost.
>
> -Taylor
>
> On Oct 6, 2016, at 6:16 PM, Joaquin Menchaca <jm...@gobalto.com>
> wrote:
>
> It might be a bug on my part, trying a test, office Internet is slooow, so
> have to step outside of office to do the test...
>
> - joaquin
>
> PS - Not on the topic of writing bugs... how could I file bugs myself? I
> wanted to make a PR for some doc bugs, but don't quite know how to PR at a
> particular tag...
>
> On Thu, Oct 6, 2016 at 1:43 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
>
>> This is a bug. The `storm list` command implementation doesn’t understand
>> the “nimbus.seeds” configuration setting.
>>
>> Can you try the following?
>>
>> Add a nimbus.host entry in your storm.yaml file with ONE of your nimbus
>> hostnames.
>>
>> Let me know if that works, and I’ll file a JIRA to get this fixed.
>>
>> -Taylor
>>
>>
>> On Oct 6, 2016, at 3:31 PM, Joaquin Menchaca <jm...@gobalto.com>
>> wrote:
>>
>> I don't get it, it seems to pick up the correct configuration... but then
>> ignores the nimbus.seeds and users localhost.
>>
>> # cat /templates/storm.yaml
>> storm.zookeeper.servers:
>> - "ip-10-110-20-8.us-west-2.compute.internal"
>>
>> nimbus.seeds: ["ip-10-110-20-7.us-west-2.compute.internal", "
>> ip-10-110-20-146.us-west-2.compute.internal"]
>>
>> # storm --config /templates/storm.yaml list
>> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name=
>> -Dstorm.options= -Dstorm.home=/usr/share/storm
>> -Dstorm.log.dir=/usr/share/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
>> *-Dstorm.conf.file=/templates/storm.yaml* -cp
>> /usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-
>> slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/
>> usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/
>> servlet-api-2.5.jar:/usr/share/storm/lib/log4j-over-
>> slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:
>> /usr/share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/
>> lib/storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm
>> -1.07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.
>> jar:/usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/
>> storm/lib/minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.
>> 10.4.jar:/templates:/usr/share/storm/bin backtype.storm.command.list
>> ...
>> 2264 [main] INFO b.s.thrift - Connecting to Nimbus at* localhost:6627 *as
>> user:
>> ...
>> Caused by: java.net.ConnectException: Connection refused
>> ...
>>
>> On Thu, Oct 6, 2016 at 12:15 PM, P. Taylor Goetz <pt...@gmail.com>
>> wrote:
>>
>>> For some reason it is trying to connect to localhost:
>>>
>>> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as
>>> user:
>>>
>>> Do you have a ls ~/.storm/storm.yaml file that might be getting picked
>>> up?
>>>
>>> What are the contents of the storm.yaml you are using in your commands?
>>>
>>> -Taylor
>>>
>>>
>>> On Oct 6, 2016, at 2:45 PM, Joaquin Menchaca <jm...@gobalto.com>
>>> wrote:
>>>
>>> I have only the nimbus.seeds configured, do I need anything else
>>> configured to get his to work?
>>>
>>> $ storm --config /templates/storm.yaml localconfvalue nimbus.seeds
>>> nimbus.seeds: [ip-10-110-20-7.us-west-2.compute.internal
>>> ip-10-110-20-146.us-west-2.compute.internal]
>>>
>>> $ nc -vz ip-10-110-20-7.us-west-2.compute.internal 6627
>>> Connection to ip-10-110-20-7.us-west-2.compute.internal 6627 port
>>> [tcp/*] succeeded!
>>> $ nc -vz ip-10-110-20-146.us-west-2.compute.internal 6627
>>> Connection to ip-10-110-20-146.us-west-2.compute.internal 6627 port
>>> [tcp/*] succeeded!
>>>
>>> $ storm --config /templates/storm.yaml list
>>> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name=
>>> -Dstorm.options= -Dstorm.home=/usr/share/storm
>>> -Dstorm.log.dir=/usr/share/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
>>> -Dstorm.conf.file=/templates/storm.yaml -cp
>>> /usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-
>>> slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/
>>> usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/servl
>>> et-api-2.5.jar:/usr/share/storm/lib/log4j-over-slf4j-1.
>>> 6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:/usr/
>>> share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/lib/
>>> storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm-1.
>>> 07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.jar:/
>>> usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/storm/lib/
>>> minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.10.4.
>>> jar:/templates:/usr/share/storm/bin backtype.storm.command.list
>>> 1283 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>>> 2263 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>>> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as
>>> user:
>>> 2293 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>>> 2348 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The
>>> baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
>>> Exception in thread "main" java.lang.RuntimeException:
>>> org.apache.thrift7.transport.TTransportException:
>>> java.net.ConnectException: Connection refused
>>> at backtype.storm.security.auth.TBackoffConnect.retryNext(TBack
>>> offConnect.java:59)
>>> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRe
>>> try(TBackoffConnect.java:51)
>>> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftCl
>>> ient.java:103)
>>> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClien
>>> t.java:72)
>>> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:69)
>>> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.c
>>> lj:75)
>>> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.c
>>> lj:72)
>>> at backtype.storm.command.list$_main.invoke(list.clj:22)
>>> at clojure.lang.AFn.applyToHelper(AFn.java:152)
>>> at clojure.lang.AFn.applyTo(AFn.java:144)
>>> at backtype.storm.command.list.main(Unknown Source)
>>> Caused by: org.apache.thrift7.transport.TTransportException:
>>> java.net.ConnectException: Connection refused
>>> at org.apache.thrift7.transport.TSocket.open(TSocket.java:187)
>>> at org.apache.thrift7.transport.TFramedTransport.open(TFramedTr
>>> ansport.java:81)
>>> at backtype.storm.security.auth.SimpleTransportPlugin.connect(S
>>> impleTransportPlugin.java:103)
>>> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRe
>>> try(TBackoffConnect.java:48)
>>> ... 9 more
>>> Caused by: java.net.ConnectException: Connection refused
>>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>>> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSock
>>> etImpl.java:350)
>>> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPl
>>> ainSocketImpl.java:206)
>>> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocket
>>> Impl.java:188)
>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>>> at java.net.Socket.connect(Socket.java:589)
>>> at org.apache.thrift7.transport.TSocket.open(TSocket.java:182)
>>> ... 12 more
>>>
>>>
>>>
>>
>>
>> --
>>
>> 是故勝兵先勝而後求戰，敗兵先戰而後求勝。
>>
>>
>>
>
>
> --
>
> 是故勝兵先勝而後求戰，敗兵先戰而後求勝。
>
>
--
是故勝兵先勝而後求戰，敗兵先戰而後求勝。
Re: Storm Bug? Cannot connect to cluster
Posted by "P. Taylor Goetz" <pt...@gmail.com>.
I don't think it's a bug on your part. I tested it and found that if only nimbus.seed is set, the 'storm list' command will attempt to connect with localhost.
-Taylor
> On Oct 6, 2016, at 6:16 PM, Joaquin Menchaca <jm...@gobalto.com> wrote:
>
> It might be a bug on my part, trying a test, office Internet is slooow, so have to step outside of office to do the test...
>
> - joaquin
>
> PS - Not on the topic of writing bugs... how could I file bugs myself? I wanted to make a PR for some doc bugs, but don't quite know how to PR at a particular tag...
>
>> On Thu, Oct 6, 2016 at 1:43 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
>> This is a bug. The `storm list` command implementation doesn’t understand the “nimbus.seeds” configuration setting.
>>
>> Can you try the following?
>>
>> Add a nimbus.host entry in your storm.yaml file with ONE of your nimbus hostnames.
>>
>> Let me know if that works, and I’ll file a JIRA to get this fixed.
>>
>> -Taylor
>>
>>
>>> On Oct 6, 2016, at 3:31 PM, Joaquin Menchaca <jm...@gobalto.com> wrote:
>>>
>>> I don't get it, it seems to pick up the correct configuration... but then ignores the nimbus.seeds and users localhost.
>>>
>>> # cat /templates/storm.yaml
>>> storm.zookeeper.servers:
>>> - "ip-10-110-20-8.us-west-2.compute.internal"
>>>
>>> nimbus.seeds: ["ip-10-110-20-7.us-west-2.compute.internal", "ip-10-110-20-146.us-west-2.compute.internal"]
>>>
>>> # storm --config /templates/storm.yaml list
>>> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/share/storm -Dstorm.log.dir=/usr/share/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=/templates/storm.yaml -cp /usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/servlet-api-2.5.jar:/usr/share/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:/usr/share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/lib/storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm-1.07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.jar:/usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/storm/lib/minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.10.4.jar:/templates:/usr/share/storm/bin backtype.storm.command.list
>>> ...
>>> 2264 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as user:
>>> ...
>>> Caused by: java.net.ConnectException: Connection refused
>>> ...
>>>
>>>> On Thu, Oct 6, 2016 at 12:15 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
>>>> For some reason it is trying to connect to localhost:
>>>>
>>>> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as user:
>>>>
>>>> Do you have a ls ~/.storm/storm.yaml file that might be getting picked up?
>>>>
>>>> What are the contents of the storm.yaml you are using in your commands?
>>>>
>>>> -Taylor
>>>>
>>>>
>>>>> On Oct 6, 2016, at 2:45 PM, Joaquin Menchaca <jm...@gobalto.com> wrote:
>>>>>
>>>>> I have only the nimbus.seeds configured, do I need anything else configured to get his to work?
>>>>>
>>>>> $ storm --config /templates/storm.yaml localconfvalue nimbus.seeds
>>>>> nimbus.seeds: [ip-10-110-20-7.us-west-2.compute.internal ip-10-110-20-146.us-west-2.compute.internal]
>>>>>
>>>>> $ nc -vz ip-10-110-20-7.us-west-2.compute.internal 6627
>>>>> Connection to ip-10-110-20-7.us-west-2.compute.internal 6627 port [tcp/*] succeeded!
>>>>> $ nc -vz ip-10-110-20-146.us-west-2.compute.internal 6627
>>>>> Connection to ip-10-110-20-146.us-west-2.compute.internal 6627 port [tcp/*] succeeded!
>>>>>
>>>>> $ storm --config /templates/storm.yaml list
>>>>> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/share/storm -Dstorm.log.dir=/usr/share/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=/templates/storm.yaml -cp /usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/servlet-api-2.5.jar:/usr/share/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:/usr/share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/lib/storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm-1.07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.jar:/usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/storm/lib/minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.10.4.jar:/templates:/usr/share/storm/bin backtype.storm.command.list
>>>>> 1283 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>>>>> 2263 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>>>>> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as user:
>>>>> 2293 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>>>>> 2348 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
>>>>> Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
>>>>> at backtype.storm.security.auth.TBackoffConnect.retryNext(TBackoffConnect.java:59)
>>>>> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:51)
>>>>> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103)
>>>>> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72)
>>>>> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:69)
>>>>> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75)
>>>>> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:72)
>>>>> at backtype.storm.command.list$_main.invoke(list.clj:22)
>>>>> at clojure.lang.AFn.applyToHelper(AFn.java:152)
>>>>> at clojure.lang.AFn.applyTo(AFn.java:144)
>>>>> at backtype.storm.command.list.main(Unknown Source)
>>>>> Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
>>>>> at org.apache.thrift7.transport.TSocket.open(TSocket.java:187)
>>>>> at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
>>>>> at backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:103)
>>>>> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
>>>>> ... 9 more
>>>>> Caused by: java.net.ConnectException: Connection refused
>>>>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
>>>>> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>>>>> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>>>>> at java.net.Socket.connect(Socket.java:589)
>>>>> at org.apache.thrift7.transport.TSocket.open(TSocket.java:182)
>>>>> ... 12 more
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> 是故勝兵先勝而後求戰，敗兵先戰而後求勝。
>>
>
>
>
> --
>
> 是故勝兵先勝而後求戰，敗兵先戰而後求勝。
Re: Storm Bug? Cannot connect to cluster
Posted by Joaquin Menchaca <jm...@gobalto.com>.
It might be a bug on my part, trying a test, office Internet is slooow, so
have to step outside of office to do the test...
- joaquin
PS - Not on the topic of writing bugs... how could I file bugs myself? I
wanted to make a PR for some doc bugs, but don't quite know how to PR at a
particular tag...
On Thu, Oct 6, 2016 at 1:43 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
> This is a bug. The `storm list` command implementation doesn’t understand
> the “nimbus.seeds” configuration setting.
>
> Can you try the following?
>
> Add a nimbus.host entry in your storm.yaml file with ONE of your nimbus
> hostnames.
>
> Let me know if that works, and I’ll file a JIRA to get this fixed.
>
> -Taylor
>
>
> On Oct 6, 2016, at 3:31 PM, Joaquin Menchaca <jm...@gobalto.com>
> wrote:
>
> I don't get it, it seems to pick up the correct configuration... but then
> ignores the nimbus.seeds and users localhost.
>
> # cat /templates/storm.yaml
> storm.zookeeper.servers:
> - "ip-10-110-20-8.us-west-2.compute.internal"
>
> nimbus.seeds: ["ip-10-110-20-7.us-west-2.compute.internal", "
> ip-10-110-20-146.us-west-2.compute.internal"]
>
> # storm --config /templates/storm.yaml list
> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name=
> -Dstorm.options= -Dstorm.home=/usr/share/storm -Dstorm.log.dir=/usr/share/storm/logs
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
> *-Dstorm.conf.file=/templates/storm.yaml* -cp
> /usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/
> log4j-slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.
> 1.jar:/usr/share/storm/lib/kryo-2.21.jar:/usr/share/
> storm/lib/servlet-api-2.5.jar:/usr/share/storm/lib/log4j-
> over-slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.
> 7.jar:/usr/share/storm/lib/clojure-1.6.0.jar:/usr/share/
> storm/lib/storm-core-0.10.0.jar:/usr/share/storm/lib/
> reflectasm-1.07-shaded.jar:/usr/share/storm/lib/hadoop-
> auth-2.4.0.jar:/usr/share/storm/lib/log4j-api-2.1.jar:/
> usr/share/storm/lib/minlog-1.2.jar:/usr/share/storm/lib/
> disruptor-2.10.4.jar:/templates:/usr/share/storm/bin
> backtype.storm.command.list
> ...
> 2264 [main] INFO b.s.thrift - Connecting to Nimbus at* localhost:6627 *as
> user:
> ...
> Caused by: java.net.ConnectException: Connection refused
> ...
>
> On Thu, Oct 6, 2016 at 12:15 PM, P. Taylor Goetz <pt...@gmail.com>
> wrote:
>
>> For some reason it is trying to connect to localhost:
>>
>> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as
>> user:
>>
>> Do you have a ls ~/.storm/storm.yaml file that might be getting picked up?
>>
>> What are the contents of the storm.yaml you are using in your commands?
>>
>> -Taylor
>>
>>
>> On Oct 6, 2016, at 2:45 PM, Joaquin Menchaca <jm...@gobalto.com>
>> wrote:
>>
>> I have only the nimbus.seeds configured, do I need anything else
>> configured to get his to work?
>>
>> $ storm --config /templates/storm.yaml localconfvalue nimbus.seeds
>> nimbus.seeds: [ip-10-110-20-7.us-west-2.compute.internal
>> ip-10-110-20-146.us-west-2.compute.internal]
>>
>> $ nc -vz ip-10-110-20-7.us-west-2.compute.internal 6627
>> Connection to ip-10-110-20-7.us-west-2.compute.internal 6627 port
>> [tcp/*] succeeded!
>> $ nc -vz ip-10-110-20-146.us-west-2.compute.internal 6627
>> Connection to ip-10-110-20-146.us-west-2.compute.internal 6627 port
>> [tcp/*] succeeded!
>>
>> $ storm --config /templates/storm.yaml list
>> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name=
>> -Dstorm.options= -Dstorm.home=/usr/share/storm
>> -Dstorm.log.dir=/usr/share/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
>> -Dstorm.conf.file=/templates/storm.yaml -cp
>> /usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-
>> slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/
>> usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/
>> servlet-api-2.5.jar:/usr/share/storm/lib/log4j-over-
>> slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:
>> /usr/share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/
>> lib/storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm
>> -1.07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.
>> jar:/usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/
>> storm/lib/minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.
>> 10.4.jar:/templates:/usr/share/storm/bin backtype.storm.command.list
>> 1283 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>> 2263 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as
>> user:
>> 2293 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>> 2348 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The
>> baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
>> Exception in thread "main" java.lang.RuntimeException:
>> org.apache.thrift7.transport.TTransportException:
>> java.net.ConnectException: Connection refused
>> at backtype.storm.security.auth.TBackoffConnect.retryNext(TBack
>> offConnect.java:59)
>> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRe
>> try(TBackoffConnect.java:51)
>> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftCl
>> ient.java:103)
>> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClien
>> t.java:72)
>> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:69)
>> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75)
>> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:72)
>> at backtype.storm.command.list$_main.invoke(list.clj:22)
>> at clojure.lang.AFn.applyToHelper(AFn.java:152)
>> at clojure.lang.AFn.applyTo(AFn.java:144)
>> at backtype.storm.command.list.main(Unknown Source)
>> Caused by: org.apache.thrift7.transport.TTransportException:
>> java.net.ConnectException: Connection refused
>> at org.apache.thrift7.transport.TSocket.open(TSocket.java:187)
>> at org.apache.thrift7.transport.TFramedTransport.open(TFramedTr
>> ansport.java:81)
>> at backtype.storm.security.auth.SimpleTransportPlugin.connect(S
>> impleTransportPlugin.java:103)
>> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRe
>> try(TBackoffConnect.java:48)
>> ... 9 more
>> Caused by: java.net.ConnectException: Connection refused
>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSock
>> etImpl.java:350)
>> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPl
>> ainSocketImpl.java:206)
>> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocket
>> Impl.java:188)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>> at java.net.Socket.connect(Socket.java:589)
>> at org.apache.thrift7.transport.TSocket.open(TSocket.java:182)
>> ... 12 more
>>
>>
>>
>
>
> --
>
> 是故勝兵先勝而後求戰，敗兵先戰而後求勝。
>
>
>
--
是故勝兵先勝而後求戰，敗兵先戰而後求勝。
Re: Storm Bug? Cannot connect to cluster
Posted by "P. Taylor Goetz" <pt...@gmail.com>.
This is a bug. The `storm list` command implementation doesn’t understand the “nimbus.seeds” configuration setting.
Can you try the following?
Add a nimbus.host entry in your storm.yaml file with ONE of your nimbus hostnames.
Let me know if that works, and I’ll file a JIRA to get this fixed.
-Taylor
> On Oct 6, 2016, at 3:31 PM, Joaquin Menchaca <jm...@gobalto.com> wrote:
>
> I don't get it, it seems to pick up the correct configuration... but then ignores the nimbus.seeds and users localhost.
>
> # cat /templates/storm.yaml
> storm.zookeeper.servers:
> - "ip-10-110-20-8.us-west-2.compute.internal"
>
> nimbus.seeds: ["ip-10-110-20-7.us-west-2.compute.internal", "ip-10-110-20-146.us-west-2.compute.internal"]
>
> # storm --config /templates/storm.yaml list
> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/share/storm -Dstorm.log.dir=/usr/share/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=/templates/storm.yaml -cp /usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/servlet-api-2.5.jar:/usr/share/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:/usr/share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/lib/storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm-1.07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.jar:/usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/storm/lib/minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.10.4.jar:/templates:/usr/share/storm/bin backtype.storm.command.list
> ...
> 2264 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as user:
> ...
> Caused by: java.net.ConnectException: Connection refused
> ...
>
> On Thu, Oct 6, 2016 at 12:15 PM, P. Taylor Goetz <ptgoetz@gmail.com <ma...@gmail.com>> wrote:
> For some reason it is trying to connect to localhost:
>
> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as user:
>
> Do you have a ls ~/.storm/storm.yaml file that might be getting picked up?
>
> What are the contents of the storm.yaml you are using in your commands?
>
> -Taylor
>
>
>> On Oct 6, 2016, at 2:45 PM, Joaquin Menchaca <jmenchaca@gobalto.com <ma...@gobalto.com>> wrote:
>>
>> I have only the nimbus.seeds configured, do I need anything else configured to get his to work?
>>
>> $ storm --config /templates/storm.yaml localconfvalue nimbus.seeds
>> nimbus.seeds: [ip-10-110-20-7.us <http://ip-10-110-20-7.us/>-west-2.compute.internal ip-10-110-20-146.us <http://ip-10-110-20-146.us/>-west-2.compute.internal]
>>
>> $ nc -vz ip-10-110-20-7.us <http://ip-10-110-20-7.us/>-west-2.compute.internal 6627
>> Connection to ip-10-110-20-7.us <http://ip-10-110-20-7.us/>-west-2.compute.internal 6627 port [tcp/*] succeeded!
>> $ nc -vz ip-10-110-20-146.us <http://ip-10-110-20-146.us/>-west-2.compute.internal 6627
>> Connection to ip-10-110-20-146.us <http://ip-10-110-20-146.us/>-west-2.compute.internal 6627 port [tcp/*] succeeded!
>>
>> $ storm --config /templates/storm.yaml list
>> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/share/storm -Dstorm.log.dir=/usr/share/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=/templates/storm.yaml -cp /usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/servlet-api-2.5.jar:/usr/share/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:/usr/share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/lib/storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm-1.07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.jar:/usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/storm/lib/minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.10.4.jar:/templates:/usr/share/storm/bin backtype.storm.command.list
>> 1283 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>> 2263 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as user:
>> 2293 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
>> 2348 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
>> Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
>> at backtype.storm.security.auth.TBackoffConnect.retryNext(TBackoffConnect.java:59)
>> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:51)
>> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103)
>> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72)
>> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:69)
>> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75)
>> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:72)
>> at backtype.storm.command.list$_main.invoke(list.clj:22)
>> at clojure.lang.AFn.applyToHelper(AFn.java:152)
>> at clojure.lang.AFn.applyTo(AFn.java:144)
>> at backtype.storm.command.list.main(Unknown Source)
>> Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
>> at org.apache.thrift7.transport.TSocket.open(TSocket.java:187)
>> at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
>> at backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:103)
>> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
>> ... 9 more
>> Caused by: java.net.ConnectException: Connection refused
>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>> at java.net <http://java.net/>.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
>> at java.net <http://java.net/>.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>> at java.net <http://java.net/>.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>> at java.net.Socket.connect(Socket.java:589)
>> at org.apache.thrift7.transport.TSocket.open(TSocket.java:182)
>> ... 12 more
>>
>
>
>
>
> --
>
> 是故勝兵先勝而後求戰，敗兵先戰而後求勝。
Re: Storm Bug? Cannot connect to cluster
Posted by Joaquin Menchaca <jm...@gobalto.com>.
I don't get it, it seems to pick up the correct configuration... but then
ignores the nimbus.seeds and users localhost.
# cat /templates/storm.yaml
storm.zookeeper.servers:
- "ip-10-110-20-8.us-west-2.compute.internal"
nimbus.seeds: ["ip-10-110-20-7.us-west-2.compute.internal",
"ip-10-110-20-146.us-west-2.compute.internal"]
# storm --config /templates/storm.yaml list
Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name=
-Dstorm.options= -Dstorm.home=/usr/share/storm
-Dstorm.log.dir=/usr/share/storm/logs
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
*-Dstorm.conf.file=/templates/storm.yaml* -cp
/usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/servlet-api-2.5.jar:/usr/share/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:/usr/share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/lib/storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm-1.07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.jar:/usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/storm/lib/minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.10.4.jar:/templates:/usr/share/storm/bin
backtype.storm.command.list
...
2264 [main] INFO b.s.thrift - Connecting to Nimbus at* localhost:6627 *as
user:
...
Caused by: java.net.ConnectException: Connection refused
...
On Thu, Oct 6, 2016 at 12:15 PM, P. Taylor Goetz <pt...@gmail.com> wrote:
> For some reason it is trying to connect to localhost:
>
> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as
> user:
>
> Do you have a ls ~/.storm/storm.yaml file that might be getting picked up?
>
> What are the contents of the storm.yaml you are using in your commands?
>
> -Taylor
>
>
> On Oct 6, 2016, at 2:45 PM, Joaquin Menchaca <jm...@gobalto.com>
> wrote:
>
> I have only the nimbus.seeds configured, do I need anything else
> configured to get his to work?
>
> $ storm --config /templates/storm.yaml localconfvalue nimbus.seeds
> nimbus.seeds: [ip-10-110-20-7.us-west-2.compute.internal
> ip-10-110-20-146.us-west-2.compute.internal]
>
> $ nc -vz ip-10-110-20-7.us-west-2.compute.internal 6627
> Connection to ip-10-110-20-7.us-west-2.compute.internal 6627 port [tcp/*]
> succeeded!
> $ nc -vz ip-10-110-20-146.us-west-2.compute.internal 6627
> Connection to ip-10-110-20-146.us-west-2.compute.internal 6627 port
> [tcp/*] succeeded!
>
> $ storm --config /templates/storm.yaml list
> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name=
> -Dstorm.options= -Dstorm.home=/usr/share/storm -Dstorm.log.dir=/usr/share/storm/logs
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
> -Dstorm.conf.file=/templates/storm.yaml -cp /usr/share/storm/lib/asm-4.0.
> jar:/usr/share/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/
> share/storm/lib/log4j-core-2.1.jar:/usr/share/storm/lib/
> kryo-2.21.jar:/usr/share/storm/lib/servlet-api-2.5.jar:
> /usr/share/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/
> share/storm/lib/slf4j-api-1.7.7.jar:/usr/share/storm/lib/
> clojure-1.6.0.jar:/usr/share/storm/lib/storm-core-0.10.0.
> jar:/usr/share/storm/lib/reflectasm-1.07-shaded.jar:/
> usr/share/storm/lib/hadoop-auth-2.4.0.jar:/usr/share/
> storm/lib/log4j-api-2.1.jar:/usr/share/storm/lib/minlog-1.
> 2.jar:/usr/share/storm/lib/disruptor-2.10.4.jar:/templates:/usr/share/storm/bin
> backtype.storm.command.list
> 1283 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
> 2263 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as
> user:
> 2293 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
> 2348 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The
> baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
> Exception in thread "main" java.lang.RuntimeException:
> org.apache.thrift7.transport.TTransportException:
> java.net.ConnectException: Connection refused
> at backtype.storm.security.auth.TBackoffConnect.retryNext(
> TBackoffConnect.java:59)
> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(
> TBackoffConnect.java:51)
> at backtype.storm.security.auth.ThriftClient.reconnect(
> ThriftClient.java:103)
> at backtype.storm.security.auth.ThriftClient.<init>(
> ThriftClient.java:72)
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:69)
> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75)
> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:72)
> at backtype.storm.command.list$_main.invoke(list.clj:22)
> at clojure.lang.AFn.applyToHelper(AFn.java:152)
> at clojure.lang.AFn.applyTo(AFn.java:144)
> at backtype.storm.command.list.main(Unknown Source)
> Caused by: org.apache.thrift7.transport.TTransportException:
> java.net.ConnectException: Connection refused
> at org.apache.thrift7.transport.TSocket.open(TSocket.java:187)
> at org.apache.thrift7.transport.TFramedTransport.open(
> TFramedTransport.java:81)
> at backtype.storm.security.auth.SimpleTransportPlugin.connect(
> SimpleTransportPlugin.java:103)
> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(
> TBackoffConnect.java:48)
> ... 9 more
> Caused by: java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(
> AbstractPlainSocketImpl.java:350)
> at java.net.AbstractPlainSocketImpl.connectToAddress(
> AbstractPlainSocketImpl.java:206)
> at java.net.AbstractPlainSocketImpl.connect(
> AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:589)
> at org.apache.thrift7.transport.TSocket.open(TSocket.java:182)
> ... 12 more
>
>
>
--
是故勝兵先勝而後求戰，敗兵先戰而後求勝。
Re: Storm Bug? Cannot connect to cluster
Posted by "P. Taylor Goetz" <pt...@gmail.com>.
For some reason it is trying to connect to localhost:
2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as user:
Do you have a ls ~/.storm/storm.yaml file that might be getting picked up?
What are the contents of the storm.yaml you are using in your commands?
-Taylor
> On Oct 6, 2016, at 2:45 PM, Joaquin Menchaca <jm...@gobalto.com> wrote:
>
> I have only the nimbus.seeds configured, do I need anything else configured to get his to work?
>
> $ storm --config /templates/storm.yaml localconfvalue nimbus.seeds
> nimbus.seeds: [ip-10-110-20-7.us-west-2.compute.internal ip-10-110-20-146.us-west-2.compute.internal]
>
> $ nc -vz ip-10-110-20-7.us-west-2.compute.internal 6627
> Connection to ip-10-110-20-7.us-west-2.compute.internal 6627 port [tcp/*] succeeded!
> $ nc -vz ip-10-110-20-146.us-west-2.compute.internal 6627
> Connection to ip-10-110-20-146.us-west-2.compute.internal 6627 port [tcp/*] succeeded!
>
> $ storm --config /templates/storm.yaml list
> Running: /usr/lib/jvm/java-8-oracle/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/share/storm -Dstorm.log.dir=/usr/share/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=/templates/storm.yaml -cp /usr/share/storm/lib/asm-4.0.jar:/usr/share/storm/lib/log4j-slf4j-impl-2.1.jar:/usr/share/storm/lib/log4j-core-2.1.jar:/usr/share/storm/lib/kryo-2.21.jar:/usr/share/storm/lib/servlet-api-2.5.jar:/usr/share/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/share/storm/lib/slf4j-api-1.7.7.jar:/usr/share/storm/lib/clojure-1.6.0.jar:/usr/share/storm/lib/storm-core-0.10.0.jar:/usr/share/storm/lib/reflectasm-1.07-shaded.jar:/usr/share/storm/lib/hadoop-auth-2.4.0.jar:/usr/share/storm/lib/log4j-api-2.1.jar:/usr/share/storm/lib/minlog-1.2.jar:/usr/share/storm/lib/disruptor-2.10.4.jar:/templates:/usr/share/storm/bin backtype.storm.command.list
> 1283 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
> 2263 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
> 2293 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as user:
> 2293 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
> 2348 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
> Exception in thread "main" java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
> at backtype.storm.security.auth.TBackoffConnect.retryNext(TBackoffConnect.java:59)
> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:51)
> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103)
> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72)
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:69)
> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:75)
> at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:72)
> at backtype.storm.command.list$_main.invoke(list.clj:22)
> at clojure.lang.AFn.applyToHelper(AFn.java:152)
> at clojure.lang.AFn.applyTo(AFn.java:144)
> at backtype.storm.command.list.main(Unknown Source)
> Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
> at org.apache.thrift7.transport.TSocket.open(TSocket.java:187)
> at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
> at backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:103)
> at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
> ... 9 more
> Caused by: java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:589)
> at org.apache.thrift7.transport.TSocket.open(TSocket.java:182)
> ... 12 more
>