You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Shouheng Yi <sh...@gmail.com> on 2014/03/20 22:44:13 UTC
Problems with Using storm-deploy on AWS
Hi all,
I'm having a lot of troubles with deploying storm on AWS. I followed
everything on https://github.com/nathanmarz/storm-deploy, but still no
luck. Can somebody help me with this?
Here are some of my specs:
- using t1.micro, RHEL 64bit (ami-a25415cb)
- $ cat ~/.pallet/config.clj
(defpallet
:services
{
:default {
:blobstore-provider "aws-s3"
:provider "aws-ec2"
:environment {:user {:username "storm"
:private-key-path
"/home/ec2-user/.ssh/xxxx.pem"
:public-key-path
"/home/ec2-user/.ssh/xxxx.pem.pub"}
:aws-user-id "xxxx-xxxx-xxxx"}
:identity "xxxxxxxxxx"
:credential "xxxxxxxxxx"
:jclouds.regions "us-east-1"
}
})
- $ cat ~/storm-deploy/conf/clusters.yaml
nimbus.image: "us-east-1/ami-a25415cb"
nimbus.hardware: "t1.micro"
supervisor.count: 2
supervisor.image: "us-east-1/ami-a25415cb"
supervisor.hardware: "t1.micro"
zookeeper.count: 1
zookeeper.image: "us-east-1/ami-a25415cb"
zookeeper.hardware: "t1.micro
Then I got these ERRORs
DEBUG provision - Finished post-configure and exec phases
INFO provision - Attaching to Available Cluster...
ERROR logging - Exception in thread "main"
ERROR logging - java.lang.AssertionError: Assert failed: (= (count
running-nodes) 1) (form-init8570048559822528595.clj:1)
ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5440)
ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5415)
ERROR logging - at clojure.lang.Compiler.load(Compiler.java:5857)
ERROR logging - at clojure.lang.Compiler.loadFile(Compiler.java:5820)
ERROR logging - at clojure.main$load_script.invoke(main.clj:221)
ERROR logging - at clojure.main$init_opt.invoke(main.clj:226)
ERROR logging - at clojure.main$initialize.invoke(main.clj:254)
ERROR logging - at clojure.main$null_opt.invoke(main.clj:279)
ERROR logging - at clojure.main$main.doInvoke(main.clj:354)
ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:422)
ERROR logging - at clojure.lang.Var.invoke(Var.java:369)
ERROR logging - at clojure.lang.AFn.applyToHelper(AFn.java:165)
ERROR logging - at clojure.lang.Var.applyTo(Var.java:482)
ERROR logging - at clojure.main.main(main.java:37)
ERROR logging - Caused by: java.lang.AssertionError: Assert failed: (=
(count running-nodes) 1)
ERROR logging - at backtype.storm.crate.storm$nimbus_ip.invoke(storm.clj:25)
ERROR logging - at
backtype.storm.crate.storm$mk_storm_yaml.invoke(storm.clj:207)
ERROR logging - at
backtype.storm.provision$sync_storm_conf_dir.invoke(provision.clj:47)
ERROR logging - at
backtype.storm.provision$attach_BANG_.invoke(provision.clj:55)
ERROR logging - at
backtype.storm.provision$start_with_nodes_BANG_.invoke(provision.clj:85)
ERROR logging - at
backtype.storm.provision$start_BANG_.invoke(provision.clj:91)
ERROR logging - at
backtype.storm.provision$_main$fn__8422.invoke(provision.clj:144)
ERROR logging - at
backtype.storm.provision$_main.doInvoke(provision.clj:130)
ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:483)
ERROR logging - at clojure.lang.Var.invoke(Var.java:381)
ERROR logging - at
user$eval5$fn__7.invoke(form-init8570048559822528595.clj:1)
ERROR logging - at user$eval5.invoke(form-init8570048559822528595.clj:1)
ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5424)
ERROR logging - ... 13 more
What is wrong with it? Can someone help me?
Thanks
Re: Problems with Using storm-deploy on AWS
Posted by Srinath C <sr...@gmail.com>.
Yes, it needs ubuntu images :)
On Fri, Mar 21, 2014 at 10:25 PM, Shouheng Yi <sh...@gmail.com> wrote:
> Hi Srinath,
>
> I have switched from t1.micro to m1.large, but got the same ERROR logging
> again. I'm using RHEL instead of ubuntu. Do you think that was the problem?
> Thank you for your timely reply :)
>
> Best,
> Shouheng
>
>
> On Thu, Mar 20, 2014 at 5:43 PM, Srinath C <sr...@gmail.com> wrote:
>
>> Yi, I haven't seen this error, but I think you definitely need a better
>> instance than t1.micro.
>> I was able to successfully get it up and running with m1.large after
>> failing to bring it up on m1.small.
>> But you could try m1.medium and see if that works for you.
>>
>>
>>
>> On Fri, Mar 21, 2014 at 3:14 AM, Shouheng Yi <sh...@gmail.com>wrote:
>>
>>> Hi all,
>>>
>>> I'm having a lot of troubles with deploying storm on AWS. I followed
>>> everything on https://github.com/nathanmarz/storm-deploy, but still no
>>> luck. Can somebody help me with this?
>>>
>>> Here are some of my specs:
>>>
>>> - using t1.micro, RHEL 64bit (ami-a25415cb)
>>>
>>> - $ cat ~/.pallet/config.clj
>>> (defpallet
>>> :services
>>> {
>>> :default {
>>> :blobstore-provider "aws-s3"
>>> :provider "aws-ec2"
>>> :environment {:user {:username "storm"
>>> :private-key-path
>>> "/home/ec2-user/.ssh/xxxx.pem"
>>> :public-key-path
>>> "/home/ec2-user/.ssh/xxxx.pem.pub"}
>>> :aws-user-id "xxxx-xxxx-xxxx"}
>>> :identity "xxxxxxxxxx"
>>> :credential "xxxxxxxxxx"
>>> :jclouds.regions "us-east-1"
>>> }
>>> })
>>>
>>> - $ cat ~/storm-deploy/conf/clusters.yaml
>>> nimbus.image: "us-east-1/ami-a25415cb"
>>> nimbus.hardware: "t1.micro"
>>>
>>> supervisor.count: 2
>>> supervisor.image: "us-east-1/ami-a25415cb"
>>> supervisor.hardware: "t1.micro"
>>>
>>> zookeeper.count: 1
>>> zookeeper.image: "us-east-1/ami-a25415cb"
>>> zookeeper.hardware: "t1.micro
>>>
>>> Then I got these ERRORs
>>> DEBUG provision - Finished post-configure and exec phases
>>> INFO provision - Attaching to Available Cluster...
>>> ERROR logging - Exception in thread "main"
>>> ERROR logging - java.lang.AssertionError: Assert failed: (= (count
>>> running-nodes) 1) (form-init8570048559822528595.clj:1)
>>> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5440)
>>> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5415)
>>> ERROR logging - at clojure.lang.Compiler.load(Compiler.java:5857)
>>> ERROR logging - at clojure.lang.Compiler.loadFile(Compiler.java:5820)
>>> ERROR logging - at clojure.main$load_script.invoke(main.clj:221)
>>> ERROR logging - at clojure.main$init_opt.invoke(main.clj:226)
>>> ERROR logging - at clojure.main$initialize.invoke(main.clj:254)
>>> ERROR logging - at clojure.main$null_opt.invoke(main.clj:279)
>>> ERROR logging - at clojure.main$main.doInvoke(main.clj:354)
>>> ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:422)
>>> ERROR logging - at clojure.lang.Var.invoke(Var.java:369)
>>> ERROR logging - at clojure.lang.AFn.applyToHelper(AFn.java:165)
>>> ERROR logging - at clojure.lang.Var.applyTo(Var.java:482)
>>> ERROR logging - at clojure.main.main(main.java:37)
>>> ERROR logging - Caused by: java.lang.AssertionError: Assert failed: (=
>>> (count running-nodes) 1)
>>> ERROR logging - at
>>> backtype.storm.crate.storm$nimbus_ip.invoke(storm.clj:25)
>>> ERROR logging - at
>>> backtype.storm.crate.storm$mk_storm_yaml.invoke(storm.clj:207)
>>> ERROR logging - at
>>> backtype.storm.provision$sync_storm_conf_dir.invoke(provision.clj:47)
>>> ERROR logging - at
>>> backtype.storm.provision$attach_BANG_.invoke(provision.clj:55)
>>> ERROR logging - at
>>> backtype.storm.provision$start_with_nodes_BANG_.invoke(provision.clj:85)
>>> ERROR logging - at
>>> backtype.storm.provision$start_BANG_.invoke(provision.clj:91)
>>> ERROR logging - at
>>> backtype.storm.provision$_main$fn__8422.invoke(provision.clj:144)
>>> ERROR logging - at
>>> backtype.storm.provision$_main.doInvoke(provision.clj:130)
>>> ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:483)
>>> ERROR logging - at clojure.lang.Var.invoke(Var.java:381)
>>> ERROR logging - at
>>> user$eval5$fn__7.invoke(form-init8570048559822528595.clj:1)
>>> ERROR logging - at user$eval5.invoke(form-init8570048559822528595.clj:1)
>>> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5424)
>>> ERROR logging - ... 13 more
>>>
>>> What is wrong with it? Can someone help me?
>>>
>>> Thanks
>>>
>>
>>
>
Re: Problems with Using storm-deploy on AWS
Posted by Shouheng Yi <sh...@gmail.com>.
Hi Srinath,
I have switched from t1.micro to m1.large, but got the same ERROR logging
again. I'm using RHEL instead of ubuntu. Do you think that was the problem?
Thank you for your timely reply :)
Best,
Shouheng
On Thu, Mar 20, 2014 at 5:43 PM, Srinath C <sr...@gmail.com> wrote:
> Yi, I haven't seen this error, but I think you definitely need a better
> instance than t1.micro.
> I was able to successfully get it up and running with m1.large after
> failing to bring it up on m1.small.
> But you could try m1.medium and see if that works for you.
>
>
>
> On Fri, Mar 21, 2014 at 3:14 AM, Shouheng Yi <sh...@gmail.com> wrote:
>
>> Hi all,
>>
>> I'm having a lot of troubles with deploying storm on AWS. I followed
>> everything on https://github.com/nathanmarz/storm-deploy, but still no
>> luck. Can somebody help me with this?
>>
>> Here are some of my specs:
>>
>> - using t1.micro, RHEL 64bit (ami-a25415cb)
>>
>> - $ cat ~/.pallet/config.clj
>> (defpallet
>> :services
>> {
>> :default {
>> :blobstore-provider "aws-s3"
>> :provider "aws-ec2"
>> :environment {:user {:username "storm"
>> :private-key-path
>> "/home/ec2-user/.ssh/xxxx.pem"
>> :public-key-path
>> "/home/ec2-user/.ssh/xxxx.pem.pub"}
>> :aws-user-id "xxxx-xxxx-xxxx"}
>> :identity "xxxxxxxxxx"
>> :credential "xxxxxxxxxx"
>> :jclouds.regions "us-east-1"
>> }
>> })
>>
>> - $ cat ~/storm-deploy/conf/clusters.yaml
>> nimbus.image: "us-east-1/ami-a25415cb"
>> nimbus.hardware: "t1.micro"
>>
>> supervisor.count: 2
>> supervisor.image: "us-east-1/ami-a25415cb"
>> supervisor.hardware: "t1.micro"
>>
>> zookeeper.count: 1
>> zookeeper.image: "us-east-1/ami-a25415cb"
>> zookeeper.hardware: "t1.micro
>>
>> Then I got these ERRORs
>> DEBUG provision - Finished post-configure and exec phases
>> INFO provision - Attaching to Available Cluster...
>> ERROR logging - Exception in thread "main"
>> ERROR logging - java.lang.AssertionError: Assert failed: (= (count
>> running-nodes) 1) (form-init8570048559822528595.clj:1)
>> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5440)
>> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5415)
>> ERROR logging - at clojure.lang.Compiler.load(Compiler.java:5857)
>> ERROR logging - at clojure.lang.Compiler.loadFile(Compiler.java:5820)
>> ERROR logging - at clojure.main$load_script.invoke(main.clj:221)
>> ERROR logging - at clojure.main$init_opt.invoke(main.clj:226)
>> ERROR logging - at clojure.main$initialize.invoke(main.clj:254)
>> ERROR logging - at clojure.main$null_opt.invoke(main.clj:279)
>> ERROR logging - at clojure.main$main.doInvoke(main.clj:354)
>> ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:422)
>> ERROR logging - at clojure.lang.Var.invoke(Var.java:369)
>> ERROR logging - at clojure.lang.AFn.applyToHelper(AFn.java:165)
>> ERROR logging - at clojure.lang.Var.applyTo(Var.java:482)
>> ERROR logging - at clojure.main.main(main.java:37)
>> ERROR logging - Caused by: java.lang.AssertionError: Assert failed: (=
>> (count running-nodes) 1)
>> ERROR logging - at
>> backtype.storm.crate.storm$nimbus_ip.invoke(storm.clj:25)
>> ERROR logging - at
>> backtype.storm.crate.storm$mk_storm_yaml.invoke(storm.clj:207)
>> ERROR logging - at
>> backtype.storm.provision$sync_storm_conf_dir.invoke(provision.clj:47)
>> ERROR logging - at
>> backtype.storm.provision$attach_BANG_.invoke(provision.clj:55)
>> ERROR logging - at
>> backtype.storm.provision$start_with_nodes_BANG_.invoke(provision.clj:85)
>> ERROR logging - at
>> backtype.storm.provision$start_BANG_.invoke(provision.clj:91)
>> ERROR logging - at
>> backtype.storm.provision$_main$fn__8422.invoke(provision.clj:144)
>> ERROR logging - at
>> backtype.storm.provision$_main.doInvoke(provision.clj:130)
>> ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:483)
>> ERROR logging - at clojure.lang.Var.invoke(Var.java:381)
>> ERROR logging - at
>> user$eval5$fn__7.invoke(form-init8570048559822528595.clj:1)
>> ERROR logging - at user$eval5.invoke(form-init8570048559822528595.clj:1)
>> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5424)
>> ERROR logging - ... 13 more
>>
>> What is wrong with it? Can someone help me?
>>
>> Thanks
>>
>
>
Re: Problems with Using storm-deploy on AWS
Posted by Srinath C <sr...@gmail.com>.
Yi, I haven't seen this error, but I think you definitely need a better
instance than t1.micro.
I was able to successfully get it up and running with m1.large after
failing to bring it up on m1.small.
But you could try m1.medium and see if that works for you.
On Fri, Mar 21, 2014 at 3:14 AM, Shouheng Yi <sh...@gmail.com> wrote:
> Hi all,
>
> I'm having a lot of troubles with deploying storm on AWS. I followed
> everything on https://github.com/nathanmarz/storm-deploy, but still no
> luck. Can somebody help me with this?
>
> Here are some of my specs:
>
> - using t1.micro, RHEL 64bit (ami-a25415cb)
>
> - $ cat ~/.pallet/config.clj
> (defpallet
> :services
> {
> :default {
> :blobstore-provider "aws-s3"
> :provider "aws-ec2"
> :environment {:user {:username "storm"
> :private-key-path
> "/home/ec2-user/.ssh/xxxx.pem"
> :public-key-path
> "/home/ec2-user/.ssh/xxxx.pem.pub"}
> :aws-user-id "xxxx-xxxx-xxxx"}
> :identity "xxxxxxxxxx"
> :credential "xxxxxxxxxx"
> :jclouds.regions "us-east-1"
> }
> })
>
> - $ cat ~/storm-deploy/conf/clusters.yaml
> nimbus.image: "us-east-1/ami-a25415cb"
> nimbus.hardware: "t1.micro"
>
> supervisor.count: 2
> supervisor.image: "us-east-1/ami-a25415cb"
> supervisor.hardware: "t1.micro"
>
> zookeeper.count: 1
> zookeeper.image: "us-east-1/ami-a25415cb"
> zookeeper.hardware: "t1.micro
>
> Then I got these ERRORs
> DEBUG provision - Finished post-configure and exec phases
> INFO provision - Attaching to Available Cluster...
> ERROR logging - Exception in thread "main"
> ERROR logging - java.lang.AssertionError: Assert failed: (= (count
> running-nodes) 1) (form-init8570048559822528595.clj:1)
> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5440)
> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5415)
> ERROR logging - at clojure.lang.Compiler.load(Compiler.java:5857)
> ERROR logging - at clojure.lang.Compiler.loadFile(Compiler.java:5820)
> ERROR logging - at clojure.main$load_script.invoke(main.clj:221)
> ERROR logging - at clojure.main$init_opt.invoke(main.clj:226)
> ERROR logging - at clojure.main$initialize.invoke(main.clj:254)
> ERROR logging - at clojure.main$null_opt.invoke(main.clj:279)
> ERROR logging - at clojure.main$main.doInvoke(main.clj:354)
> ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:422)
> ERROR logging - at clojure.lang.Var.invoke(Var.java:369)
> ERROR logging - at clojure.lang.AFn.applyToHelper(AFn.java:165)
> ERROR logging - at clojure.lang.Var.applyTo(Var.java:482)
> ERROR logging - at clojure.main.main(main.java:37)
> ERROR logging - Caused by: java.lang.AssertionError: Assert failed: (=
> (count running-nodes) 1)
> ERROR logging - at
> backtype.storm.crate.storm$nimbus_ip.invoke(storm.clj:25)
> ERROR logging - at
> backtype.storm.crate.storm$mk_storm_yaml.invoke(storm.clj:207)
> ERROR logging - at
> backtype.storm.provision$sync_storm_conf_dir.invoke(provision.clj:47)
> ERROR logging - at
> backtype.storm.provision$attach_BANG_.invoke(provision.clj:55)
> ERROR logging - at
> backtype.storm.provision$start_with_nodes_BANG_.invoke(provision.clj:85)
> ERROR logging - at
> backtype.storm.provision$start_BANG_.invoke(provision.clj:91)
> ERROR logging - at
> backtype.storm.provision$_main$fn__8422.invoke(provision.clj:144)
> ERROR logging - at
> backtype.storm.provision$_main.doInvoke(provision.clj:130)
> ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:483)
> ERROR logging - at clojure.lang.Var.invoke(Var.java:381)
> ERROR logging - at
> user$eval5$fn__7.invoke(form-init8570048559822528595.clj:1)
> ERROR logging - at user$eval5.invoke(form-init8570048559822528595.clj:1)
> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5424)
> ERROR logging - ... 13 more
>
> What is wrong with it? Can someone help me?
>
> Thanks
>