You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Shouheng Yi <sh...@gmail.com> on 2014/03/19 22:26:55 UTC
Problems for Deploying Storm to EC2
Hi all,
I'm very new to Storm and I wanted to deploy Storm on a free AWS instance
by using storm-deploy (https://github.com/nathanmarz/storm-deploy/wiki). I
followed the instructions on the link but it still gives me error.
Here is my ~.pallet.config.clj
(defpallet
:services
{
:default {
:blobstore-provider "aws-s3"
:provider "aws-ec2"
:environment {:user {:username "storm" ; this must be "storm"
:private-key-path
"/home/ec2-user/xxxx.pem"
:public-key-path
"/home/ec2-user/xxxx.pem.pub"}
:aws-user-id "xxxx-xxxx-xxxx"}
:identity "xxxxxxxx"
:credential "xxxxxxxx"
:jclouds.regions "us-east-1c"
}
})
Here is my clusters.yaml
nimbus.image: "us-east-1c/ami-a25415cb"
nimbus.hardware: "t1.micro"
supervisor.count: 3
supervisor.image: "us-east-1c/ami-a25415cb"
supervisor.hardware: "t1.micro"
#supervisor.spot.price: 1.60
zookeeper.count: 1
zookeeper.image: "us-east-1c/ami-a25415cb"
zookeeper.hardware: "t1.micro"
Then I run
$ lein deploy-storm --start --name myuster --branch master > ~/out.txt
It gave me:
INFO execute - Output:
/home/ec2-user/mephyi.pem.pub
DEBUG execute - out
=> "/home/ec2-user/mephyi.pem.pub\n"
INFO execute - Output:
/home/ec2-user/mephyi.pem
DEBUG execute - out
=> "/home/ec2-user/mephyi.pem\n"
INFO execute - Output:
storm
DEBUG execute - out
=> "storm\n"
INFO execute - Output:
/home/ec2-user/mephyi.pem.pub
DEBUG execute - out
=> "/home/ec2-user/mephyi.pem.pub\n"
INFO execute - Output:
/home/ec2-user/mephyi.pem
DEBUG execute - out
=> "/home/ec2-user/mephyi.pem\n"
INFO execute - Output:
/home/ec2-user/mephyi.pem.pub
DEBUG execute - out
=> "/home/ec2-user/mephyi.pem.pub\n"
INFO execute - Output:
/home/ec2-user/mephyi.pem
DEBUG execute - out
=> "/home/ec2-user/mephyi.pem\n"
DEBUG jclouds - Found jclouds sshj driver
DEBUG jclouds - extensions (:log4j :slf4j :sshj)
DEBUG jclouds - options [:jclouds.regions "us-east-1c" :blobstore-provider
"aws-s3"]
ERROR logging - Exception in thread "main"
ERROR logging - com.google.inject.CreationException: Guice creation errors:
1) org.jclouds.rest.RestContext<org.jclouds.aws.ec2.AWSEC2Client, A> cannot
be used as a key; It is not fully specified.
1 error (form-init3235628430015742355.clj:1)
ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5440)
ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5415)
ERROR logging - at clojure.lang.Compiler.load(Compiler.java:5857)
ERROR logging - at clojure.lang.Compiler.loadFile(Compiler.java:5820)
ERROR logging - at clojure.main$load_script.invoke(main.clj:221)
ERROR logging - at clojure.main$init_opt.invoke(main.clj:226)
ERROR logging - at clojure.main$initialize.invoke(main.clj:254)
ERROR logging - at clojure.main$null_opt.invoke(main.clj:279)
ERROR logging - at clojure.main$main.doInvoke(main.clj:354)
ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:422)
ERROR logging - at clojure.lang.Var.invoke(Var.java:369)
ERROR logging - at clojure.lang.AFn.applyToHelper(AFn.java:165)
ERROR logging - at clojure.lang.Var.applyTo(Var.java:482)
ERROR logging - at clojure.main.main(main.java:37)
ERROR logging - Caused by: com.google.inject.CreationException: Guice
creation errors:
1) org.jclouds.rest.RestContext<org.jclouds.aws.ec2.AWSEC2Client, A> cannot
be used as a key; It is not fully specified.
1 error
ERROR logging - at
com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:435)
ERROR logging - at
com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:154)
ERROR logging - at
com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:106)
ERROR logging - at com.google.inject.Guice.createInjector(Guice.java:95)
ERROR logging - at
org.jclouds.ContextBuilder.buildInjector(ContextBuilder.java:324)
ERROR logging - at
org.jclouds.ContextBuilder.buildInjector(ContextBuilder.java:262)
ERROR logging - at
org.jclouds.ContextBuilder.buildView(ContextBuilder.java:524)
ERROR logging - at
org.jclouds.ContextBuilder.buildView(ContextBuilder.java:504)
ERROR logging - at
org.jclouds.compute2$compute_service.doInvoke(compute2.clj:92)
ERROR logging - at clojure.lang.RestFn.applyTo(RestFn.java:147)
ERROR logging - at clojure.core$apply.doInvoke(core.clj:548)
ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:562)
ERROR logging - at
pallet.compute.jclouds$eval5952$fn__5954.invoke(jclouds.clj:720)
ERROR logging - at clojure.lang.MultiFn.invoke(MultiFn.java:167)
ERROR logging - at pallet.compute$compute_service.doInvoke(compute.clj:36)
ERROR logging - at clojure.lang.RestFn.applyTo(RestFn.java:140)
ERROR logging - at clojure.core$apply.invoke(core.clj:542)
ERROR logging - at
pallet.configure$compute_service_from_map.invoke(configure.clj:176)
ERROR logging - at clojure.lang.Var.invoke(Var.java:365)
ERROR logging - at clojure.lang.AFn.applyToHelper(AFn.java:163)
ERROR logging - at clojure.lang.Var.applyTo(Var.java:482)
ERROR logging - at clojure.core$apply.invoke(core.clj:540)
ERROR logging - at
pallet.compute$compute_service_from_map.doInvoke(compute.clj:185)
ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:409)
ERROR logging - at backtype.storm.provision$mk_aws.invoke(provision.clj:120)
ERROR logging - at
backtype.storm.provision$_main.doInvoke(provision.clj:123)
ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:483)
ERROR logging - at clojure.lang.Var.invoke(Var.java:381)
ERROR logging - at
user$eval5$fn__7.invoke(form-init3235628430015742355.clj:1)
ERROR logging - at user$eval5.invoke(form-init3235628430015742355.clj:1)
ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5424)
ERROR logging - ... 13 more
Any ideas?
Thanks
Re: Problems for Deploying Storm to EC2
Posted by "Cody A. Ray" <co...@gmail.com>.
Just saw this: https://github.com/nathanmarz/storm-deploy/issues/62
I tried the 'update-clojure-jclouds' branch but hit a brick wall with AWS
responding with a 500 Unexpected Server Error with no details. Maybe you'll
have better luck. :)
-Cody
On Wed, Mar 19, 2014 at 4:26 PM, Shouheng Yi <sh...@gmail.com> wrote:
> Hi all,
>
> I'm very new to Storm and I wanted to deploy Storm on a free AWS instance
> by using storm-deploy (https://github.com/nathanmarz/storm-deploy/wiki).
> I followed the instructions on the link but it still gives me error.
>
> Here is my ~.pallet.config.clj
>
> (defpallet
> :services
> {
> :default {
> :blobstore-provider "aws-s3"
> :provider "aws-ec2"
> :environment {:user {:username "storm" ; this must be "storm"
> :private-key-path
> "/home/ec2-user/xxxx.pem"
> :public-key-path
> "/home/ec2-user/xxxx.pem.pub"}
> :aws-user-id "xxxx-xxxx-xxxx"}
> :identity "xxxxxxxx"
> :credential "xxxxxxxx"
> :jclouds.regions "us-east-1c"
> }
> })
>
> Here is my clusters.yaml
>
> nimbus.image: "us-east-1c/ami-a25415cb"
> nimbus.hardware: "t1.micro"
>
> supervisor.count: 3
> supervisor.image: "us-east-1c/ami-a25415cb"
> supervisor.hardware: "t1.micro"
> #supervisor.spot.price: 1.60
>
>
> zookeeper.count: 1
> zookeeper.image: "us-east-1c/ami-a25415cb"
> zookeeper.hardware: "t1.micro"
>
> Then I run
> $ lein deploy-storm --start --name myuster --branch master > ~/out.txt
>
> It gave me:
> INFO execute - Output:
> /home/ec2-user/mephyi.pem.pub
>
> DEBUG execute - out
> => "/home/ec2-user/mephyi.pem.pub\n"
> INFO execute - Output:
> /home/ec2-user/mephyi.pem
>
> DEBUG execute - out
> => "/home/ec2-user/mephyi.pem\n"
> INFO execute - Output:
> storm
>
> DEBUG execute - out
> => "storm\n"
> INFO execute - Output:
> /home/ec2-user/mephyi.pem.pub
>
> DEBUG execute - out
> => "/home/ec2-user/mephyi.pem.pub\n"
> INFO execute - Output:
> /home/ec2-user/mephyi.pem
>
> DEBUG execute - out
> => "/home/ec2-user/mephyi.pem\n"
> INFO execute - Output:
> /home/ec2-user/mephyi.pem.pub
>
> DEBUG execute - out
> => "/home/ec2-user/mephyi.pem.pub\n"
> INFO execute - Output:
> /home/ec2-user/mephyi.pem
>
> DEBUG execute - out
> => "/home/ec2-user/mephyi.pem\n"
> DEBUG jclouds - Found jclouds sshj driver
> DEBUG jclouds - extensions (:log4j :slf4j :sshj)
> DEBUG jclouds - options [:jclouds.regions "us-east-1c" :blobstore-provider
> "aws-s3"]
> ERROR logging - Exception in thread "main"
> ERROR logging - com.google.inject.CreationException: Guice creation errors:
>
> 1) org.jclouds.rest.RestContext<org.jclouds.aws.ec2.AWSEC2Client, A>
> cannot be used as a key; It is not fully specified.
>
> 1 error (form-init3235628430015742355.clj:1)
> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5440)
> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5415)
> ERROR logging - at clojure.lang.Compiler.load(Compiler.java:5857)
> ERROR logging - at clojure.lang.Compiler.loadFile(Compiler.java:5820)
> ERROR logging - at clojure.main$load_script.invoke(main.clj:221)
> ERROR logging - at clojure.main$init_opt.invoke(main.clj:226)
> ERROR logging - at clojure.main$initialize.invoke(main.clj:254)
> ERROR logging - at clojure.main$null_opt.invoke(main.clj:279)
> ERROR logging - at clojure.main$main.doInvoke(main.clj:354)
> ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:422)
> ERROR logging - at clojure.lang.Var.invoke(Var.java:369)
> ERROR logging - at clojure.lang.AFn.applyToHelper(AFn.java:165)
> ERROR logging - at clojure.lang.Var.applyTo(Var.java:482)
> ERROR logging - at clojure.main.main(main.java:37)
> ERROR logging - Caused by: com.google.inject.CreationException: Guice
> creation errors:
>
> 1) org.jclouds.rest.RestContext<org.jclouds.aws.ec2.AWSEC2Client, A>
> cannot be used as a key; It is not fully specified.
>
> 1 error
> ERROR logging - at
> com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:435)
> ERROR logging - at
> com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:154)
> ERROR logging - at
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:106)
> ERROR logging - at com.google.inject.Guice.createInjector(Guice.java:95)
> ERROR logging - at
> org.jclouds.ContextBuilder.buildInjector(ContextBuilder.java:324)
> ERROR logging - at
> org.jclouds.ContextBuilder.buildInjector(ContextBuilder.java:262)
> ERROR logging - at
> org.jclouds.ContextBuilder.buildView(ContextBuilder.java:524)
> ERROR logging - at
> org.jclouds.ContextBuilder.buildView(ContextBuilder.java:504)
> ERROR logging - at
> org.jclouds.compute2$compute_service.doInvoke(compute2.clj:92)
> ERROR logging - at clojure.lang.RestFn.applyTo(RestFn.java:147)
> ERROR logging - at clojure.core$apply.doInvoke(core.clj:548)
> ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:562)
> ERROR logging - at
> pallet.compute.jclouds$eval5952$fn__5954.invoke(jclouds.clj:720)
> ERROR logging - at clojure.lang.MultiFn.invoke(MultiFn.java:167)
> ERROR logging - at pallet.compute$compute_service.doInvoke(compute.clj:36)
> ERROR logging - at clojure.lang.RestFn.applyTo(RestFn.java:140)
> ERROR logging - at clojure.core$apply.invoke(core.clj:542)
> ERROR logging - at
> pallet.configure$compute_service_from_map.invoke(configure.clj:176)
> ERROR logging - at clojure.lang.Var.invoke(Var.java:365)
> ERROR logging - at clojure.lang.AFn.applyToHelper(AFn.java:163)
> ERROR logging - at clojure.lang.Var.applyTo(Var.java:482)
> ERROR logging - at clojure.core$apply.invoke(core.clj:540)
> ERROR logging - at
> pallet.compute$compute_service_from_map.doInvoke(compute.clj:185)
> ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:409)
> ERROR logging - at
> backtype.storm.provision$mk_aws.invoke(provision.clj:120)
> ERROR logging - at
> backtype.storm.provision$_main.doInvoke(provision.clj:123)
> ERROR logging - at clojure.lang.RestFn.invoke(RestFn.java:483)
> ERROR logging - at clojure.lang.Var.invoke(Var.java:381)
> ERROR logging - at
> user$eval5$fn__7.invoke(form-init3235628430015742355.clj:1)
> ERROR logging - at user$eval5.invoke(form-init3235628430015742355.clj:1)
> ERROR logging - at clojure.lang.Compiler.eval(Compiler.java:5424)
> ERROR logging - ... 13 more
>
>
> Any ideas?
> Thanks
>
--
Cody A. Ray, LEED AP
cody.a.ray@gmail.com
215.501.7891