You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@whirr.apache.org by Ashish <pa...@gmail.com> on 2011/01/28 09:53:13 UTC
Error while running cassandra using whirr
Folks,
I followed the instructions from
http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
Using whirr-0.2.0-incubating stable release
Instances are launched, but at the end it it displays an error while
connecting it.
Following exception is printed -----------------
Authorizing firewall
Running configuration script
Exception in thread "main" java.io.IOException:
org.jclouds.compute.RunScriptOnNodesException: error runScript on
filter
ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
Execution failures:
0 error[s]
Node failures:
1) SshException on node us-east-1/i-17f3497b:
org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
connecting to session.
at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: com.jcraft.jsch.JSchException: Auth fail
at com.jcraft.jsch.Session.connect(Session.java:452)
at com.jcraft.jsch.Session.connect(Session.java:150)
at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
... 7 more
Hadoop cluster runs fine with the whirr release.
My configuration file is plan simple, picked from the blog and the
real value replaced.
Am I missing anything out here?
thanks
ashish
Re: Error while running cassandra using whirr
Posted by Ashish <pa...@gmail.com>.
I test again with 0.3 release, and the error is same
Any clue on what might be incorrect in my configuration?
>From the stacktrace, the client is trying sftp, should it try scp?
Here are the detailed logs
2011-02-01 08:52:17,584 DEBUG [jclouds.compute] (pool-1-thread-1) <<
started instances(i-9357d9ff,i-6d54da01,i-6f54da03)
2011-02-01 08:52:18,784 DEBUG [jclouds.compute] (pool-1-thread-1) <<
present instances(i-9357d9ff,i-6d54da01,i-6f54da03)
2011-02-01 08:53:02,465 DEBUG [jclouds.compute] (user thread 5) >>
authorizing rsa public key for ubuntu@75.101.178.157
2011-02-01 08:53:03,327 DEBUG [jclouds.compute] (user thread 3) <<
initialized(0)
2011-02-01 08:53:03,328 DEBUG [jclouds.compute] (user thread 3) >>
running [sudo ./runscript start] as ubuntu@75.101.178.157
2011-02-01 08:53:03,669 DEBUG [jclouds.compute] (user thread 5) << complete(0)
2011-02-01 08:53:05,039 DEBUG [jclouds.compute] (user thread 3) << start(0)
2011-02-01 08:53:05,488 DEBUG [jclouds.compute] (user thread 11) >>
authorizing rsa public key for ubuntu@50.16.32.230
2011-02-01 08:53:07,227 DEBUG [jclouds.compute] (user thread 10) <<
initialized(0)
2011-02-01 08:53:07,227 DEBUG [jclouds.compute] (user thread 10) >>
running [sudo ./runscript start] as ubuntu@50.16.32.230
2011-02-01 08:53:07,362 DEBUG [jclouds.compute] (user thread 11) << complete(0)
2011-02-01 08:53:08,966 DEBUG [jclouds.compute] (user thread 10) << start(0)
2011-02-01 08:53:35,908 DEBUG [jclouds.compute] (user thread 3) <<
complete(true)
2011-02-01 08:53:36,779 DEBUG [jclouds.compute] (user thread 3) <<
stdout from runscript as ubuntu@75.101.178.157
update-alternatives: using /usr/lib/jvm/java-6-sun/jre/bin/orbd to
provide /usr/bin/orbd (orbd) in auto mode.
update-alternatives: using /usr/lib/jvm/java-6-sun/jre/bin/servertool
to provide /usr/bin/servertool (servertool) in auto mode.
update-alternatives: using /usr/lib/jvm/java-6-sun/jre/bin/tnameserv
to provide /usr/bin/tnameserv (tnameserv) in auto mode.
update-alternatives: using /usr/lib/jvm/java-6-sun/jre/lib/jexec to
provide /usr/bin/jexec (jexec) in auto mode.
Setting up sun-java6-jdk (6.22-0ubuntu1~10.04) ...
Setting up visualvm (1.2.2-0ubuntu2) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
2011-02-01 08:53:37,600 DEBUG [jclouds.compute] (user thread 3) <<
stderr from runscript as ubuntu@75.101.178.157
+ mkdir -p /mnt/cassandra/logs
+ ln -s /mnt/cassandra/logs /var/log/cassandra
+ mkdir -p /etc/cassandra/conf
+ cp /usr/local/apache-cassandra-0.7.0/conf/log4j-server.properties
/usr/local/apache-cassandra-0.7.0/conf/log4j-tools.properties
/etc/cassandra/conf
+ [[ 0.6 == \0\.\7 ]]
+ cp /usr/local/apache-cassandra-0.7.0/conf/cassandra.yaml /etc/cassandra/conf
+ cp /usr/local/apache-cassandra-0.7.0/conf/cassandra-env.sh /etc/cassandra/conf
+ sed -i -e 's|CASSANDRA_CONF=$CASSANDRA_HOME/conf|CASSANDRA_CONF=/etc/cassandra/conf|'
/usr/local/apache-cassandra-0.7.0/bin/cassandra.in.sh
+ sed -i -e 's/exit 0//' /etc/rc.local
+ cat
2011-02-01 08:53:40,023 ERROR [jclouds.compute] (user thread 13) ssh,
completed: 0/2, errors: 1, rate: 3272ms/op
java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
ubuntu@50.16.175.249:22: Error putting path: runscript
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.jclouds.ssh.SshException: ubuntu@50.16.175.249:22:
Error putting path: runscript
at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:94)
at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:53)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: 4: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
... 7 more
Caused by: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
at com.jcraft.jsch.ChannelSftp.checkStatus(ChannelSftp.java:1924)
at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:567)
... 10 more
2011-02-01 08:53:40,023 ERROR [jclouds.compute] (user thread 9) ssh,
completed: 0/2, errors: 2, rate: 3273ms/op
java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
ubuntu@50.16.175.249:22: Error putting path: .ssh/id_rsa.pub
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.jclouds.ssh.SshException: ubuntu@50.16.175.249:22:
Error putting path: .ssh/id_rsa.pub
at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: 4:
at com.jcraft.jsch.ChannelSftp.getHome(ChannelSftp.java:1894)
at com.jcraft.jsch.ChannelSftp.getCwd(ChannelSftp.java:1903)
at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2360)
at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:418)
at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
... 7 more
Caused by: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1810)
at com.jcraft.jsch.ChannelSftp.getHome(ChannelSftp.java:1888)
... 12 more
2011-02-01 08:53:40,037 ERROR [jclouds.compute] (user thread 1) ssh,
completed: 0/2, errors: 2, rate: 3280ms/op
java.lang.RuntimeException: ssh, completed: 0/2, errors: 2, rate: 3280ms/op
at org.jclouds.concurrent.FutureIterables.awaitCompletion(FutureIterables.java:139)
at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:245)
at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
2011-02-01 08:53:40,039 ERROR [jclouds.compute] (user thread 1) <<
problem applying options to node(us-east-1/i-6f54da03):
java.lang.RuntimeException: error invoking callables on nodes:
{org.jclouds.compute.callables.AuthorizeRSAPublicKey@1a0283e=java.util.concurrent.ExecutionException:
org.jclouds.ssh.SshException: ubuntu@50.16.175.249:22: Error putting
path: .ssh/id_rsa.pub,
org.jclouds.compute.callables.RunScriptOnNode@31f9b804=java.util.concurrent.ExecutionException:
org.jclouds.ssh.SshException: ubuntu@50.16.175.249:22: Error putting
path: runscript}
at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
2011-02-01 08:53:40,767 DEBUG [jclouds.compute] (user thread 2) >>
installing rsa key for ubuntu@75.101.178.157
2011-02-01 08:53:41,703 DEBUG [jclouds.compute] (user thread 2) <<
options applied node(us-east-1/i-9357d9ff)
2011-02-01 08:53:42,069 DEBUG [jclouds.compute] (user thread 10) <<
complete(true)
2011-02-01 08:53:43,129 DEBUG [jclouds.compute] (user thread 10) <<
stdout from runscript as ubuntu@50.16.32.230
update-alternatives: using /usr/lib/jvm/java-6-sun/jre/bin/orbd to
provide /usr/bin/orbd (orbd) in auto mode.
update-alternatives: using /usr/lib/jvm/java-6-sun/jre/bin/servertool
to provide /usr/bin/servertool (servertool) in auto mode.
update-alternatives: using /usr/lib/jvm/java-6-sun/jre/bin/tnameserv
to provide /usr/bin/tnameserv (tnameserv) in auto mode.
update-alternatives: using /usr/lib/jvm/java-6-sun/jre/lib/jexec to
provide /usr/bin/jexec (jexec) in auto mode.
Setting up sun-java6-jdk (6.22-0ubuntu1~10.04) ...
Setting up visualvm (1.2.2-0ubuntu2) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
2011-02-01 08:53:44,153 DEBUG [jclouds.compute] (user thread 10) <<
stderr from runscript as ubuntu@50.16.32.230
+ mkdir -p /mnt/cassandra/logs
+ ln -s /mnt/cassandra/logs /var/log/cassandra
+ mkdir -p /etc/cassandra/conf
+ cp /usr/local/apache-cassandra-0.7.0/conf/log4j-server.properties
/usr/local/apache-cassandra-0.7.0/conf/log4j-tools.properties
/etc/cassandra/conf
+ [[ 0.6 == \0\.\7 ]]
+ cp /usr/local/apache-cassandra-0.7.0/conf/cassandra.yaml /etc/cassandra/conf
+ cp /usr/local/apache-cassandra-0.7.0/conf/cassandra-env.sh /etc/cassandra/conf
+ sed -i -e 's|CASSANDRA_CONF=$CASSANDRA_HOME/conf|CASSANDRA_CONF=/etc/cassandra/conf|'
/usr/local/apache-cassandra-0.7.0/bin/cassandra.in.sh
+ sed -i -e 's/exit 0//' /etc/rc.local
+ cat
2011-02-01 08:53:47,843 DEBUG [jclouds.compute] (user thread 4) >>
installing rsa key for ubuntu@50.16.32.230
2011-02-01 08:53:48,628 DEBUG [jclouds.compute] (user thread 4) <<
options applied node(us-east-1/i-6d54da01)
cheers
ashish
On Sun, Jan 30, 2011 at 10:22 AM, Ashish <pa...@gmail.com> wrote:
> On Sat, Jan 29, 2011 at 10:51 PM, Lars George <la...@gmail.com> wrote:
>> Hi Ashish,
>>
>> I meant for you to use the hardware-id, image-id and location only in your config. The config is indeed for trunk but that is a different test altogether.
>>
>> Lars
>
> Hi Lars,
>
> Same errror :(
>
> 0.3 has already passed the vote, so let me wait for it and bug you
> folks once again with whatever the outcome is :)
>
> Once it works out, it would be a time to blog about it.
>
> I have a crazy idea of integrating whirr to cassandra-gui for easily
> managing the cluster. Will take a while but I think I will
> pick it up once am done porting it to cassandra 0.7
>
Re: Error while running cassandra using whirr
Posted by Ashish <pa...@gmail.com>.
On Sat, Jan 29, 2011 at 10:51 PM, Lars George <la...@gmail.com> wrote:
> Hi Ashish,
>
> I meant for you to use the hardware-id, image-id and location only in your config. The config is indeed for trunk but that is a different test altogether.
>
> Lars
Hi Lars,
Same errror :(
0.3 has already passed the vote, so let me wait for it and bug you
folks once again with whatever the outcome is :)
Once it works out, it would be a time to blog about it.
I have a crazy idea of integrating whirr to cassandra-gui for easily
managing the cluster. Will take a while but I think I will
pick it up once am done porting it to cassandra 0.7
cheers
ashish
Re: Error while running cassandra using whirr
Posted by Lars George <la...@gmail.com>.
Hi Ashish,
I meant for you to use the hardware-id, image-id and location only in your config. The config is indeed for trunk but that is a different test altogether.
Lars
On Jan 29, 2011, at 17:48, Ashish <pa...@gmail.com> wrote:
> @lars - hbase service is not part of 0.2.0. Let me try with trunk build.
>
> thanks
> ashish
>
> On Sat, Jan 29, 2011 at 10:06 PM, Ashish <pa...@gmail.com> wrote:
>> On Sat, Jan 29, 2011 at 9:55 PM, Lars George <la...@gmail.com> wrote:
>>> Hi Ashish,
>>>
>>> Cool, and you have stored them in the default location in your home dir? In other words you are sure they get picked up?
>>
>> yes they are placed in default location. Not sure if they are picked
>> up, nothing shows up in logs :(
>>
>> AMI's run successfully, its later that this problem happens.
>>
>>>
>>> Sometimes I had issues using the default AMIs and instance size, could you try what I am using here
>>>
>>> https://github.com/larsgeorge/whirr/blob/WHIRR-201/services/hbase/src/test/resources/whirr-hbase-test.properties#L27
>>>
>>> That is the Alestic AMI with Ubuntu Lucid and at least a medium if not large instance size. If that works you know at least it should work with the defaults.
>>
>> Let me try with this configuration. Hadoop cluster did ran fine, so am
>> suspecting some goofup at my end.
>>
>> Will revert back with the test result ASAP.
>>
>>>
>>> Lars
>>>
>>> On Jan 29, 2011, at 17:01, Ashish <pa...@gmail.com> wrote:
>>>
>>>> @Andrei - using 0.2.0, tried with trunk as well
>>>>
>>>> @Lars - Bulls eye, I am using the Amazon keys :(
>>>> Never crossed my mind that I might need to generate and use the keys
>>>>
>>>> Since I had a clean machine, I created the ssh keys using the command
>>>> u specified
>>>>
>>>> $ ssh-keygen -t rsa
>>>>
>>>> Then commented the properties related to keys and ran again. I ran
>>>> into another error :(
>>>>
>>>> Do you think I should move to 0.3.0 and test?
>>>>
>>>> Sorry for eating up your weekend.
>>>>
>>>> Traces below
>>>> --------
>>>>
>>>> Configuring template
>>>> Starting 3 node(s)
>>>> ssh, completed: 0/2, errors: 1, rate: 2802ms/op
>>>> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
>>>> ec2-user@184.73.14.241:22: Error connecting to sftp.
>>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>> Caused by: org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22:
>>>> Error connecting to sftp.
>>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:162)
>>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
>>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> ... 3 more
>>>> Caused by: com.jcraft.jsch.JSchException: java.io.IOException:
>>>> inputstream is closed
>>>> at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:242)
>>>> at com.jcraft.jsch.Channel.connect(Channel.java:200)
>>>> at com.jcraft.jsch.Channel.connect(Channel.java:144)
>>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:160)
>>>> ... 7 more
>>>> Caused by: java.io.IOException: inputstream is closed
>>>> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
>>>> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
>>>> at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:211)
>>>> ... 10 more
>>>> ssh, completed: 0/2, errors: 2, rate: 2803ms/op
>>>> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
>>>> ec2-user@184.73.14.241:22: Error putting path: runscript
>>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>> Caused by: org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22:
>>>> Error putting path: runscript
>>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
>>>> at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:94)
>>>> at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:53)
>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> ... 3 more
>>>> Caused by: 4: java.io.IOException: inputstream is closed
>>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
>>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
>>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
>>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
>>>> ... 7 more
>>>> Caused by: java.io.IOException: inputstream is closed
>>>> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
>>>> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
>>>> at com.jcraft.jsch.ChannelSftp.checkStatus(ChannelSftp.java:1924)
>>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:567)
>>>> ... 10 more
>>>> ssh, completed: 0/2, errors: 2, rate: 2807ms/op
>>>> java.lang.RuntimeException: ssh, completed: 0/2, errors: 2, rate: 2807ms/op
>>>> at org.jclouds.concurrent.FutureIterables.awaitCompletion(FutureIterables.java:139)
>>>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:245)
>>>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>>>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>>>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>> << problem applying options to node(us-east-1/i-831197ef):
>>>> java.lang.RuntimeException: error invoking callables on nodes:
>>>> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
>>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
>>>> path: runscript,
>>>> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
>>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
>>>> connecting to sftp.}
>>>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
>>>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>>>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>>>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>> Exception in thread "main" java.io.IOException:
>>>> org.jclouds.compute.RunNodesException: error running 3 nodes
>>>> tag(mycassandracluster) location(us-east-1) image(ami-74f0061d)
>>>> size(t1.micro) options([groupIds=[], keyPair=null, noKeyPair=false,
>>>> placementGroup=null, noPlacementGroup=false, monitoringEnabled=false,
>>>> inboundPorts=[22], privateKey=true, publicKey=true, runScript=true,
>>>> port:seconds=-1:-1, subnetId=null, metadata/details: false])
>>>> Execution failures:
>>>>
>>>> 0 error[s]
>>>> Node failures:
>>>>
>>>> 1) RuntimeException on node us-east-1/i-831197ef:
>>>> java.lang.RuntimeException: error invoking callables on nodes:
>>>> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
>>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
>>>> path: runscript,
>>>> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
>>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
>>>> connecting to sftp.}
>>>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
>>>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>>>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>>>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>>
>>>>
>>>> 1 error[s]
>>>> at org.apache.whirr.service.cassandra.CassandraService.launchCluster(CassandraService.java:114)
>>>> at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:61)
>>>> at org.apache.whirr.cli.Main.run(Main.java:65)
>>>> at org.apache.whirr.cli.Main.main(Main.java:91)
>>>> Caused by: org.jclouds.compute.RunNodesException: error running 3
>>>> nodes tag(mycassandracluster) location(us-east-1) image(ami-74f0061d)
>>>> size(t1.micro) options([groupIds=[], keyPair=null, noKeyPair=false,
>>>> placementGroup=null, noPlacementGroup=false, monitoringEnabled=false,
>>>> inboundPorts=[22], privateKey=true, publicKey=true, runScript=true,
>>>> port:seconds=-1:-1, subnetId=null, metadata/details: false])
>>>> Execution failures:
>>>>
>>>> 0 error[s]
>>>> Node failures:
>>>>
>>>> 1) RuntimeException on node us-east-1/i-831197ef:
>>>> java.lang.RuntimeException: error invoking callables on nodes:
>>>> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
>>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
>>>> path: runscript,
>>>> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
>>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
>>>> connecting to sftp.}
>>>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
>>>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>>>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>>>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>>
>>>>
>>>> 1 error[s]
>>>> at org.jclouds.compute.internal.BaseComputeService.runNodesWithTag(BaseComputeService.java:162)
>>>> at org.apache.whirr.service.cassandra.CassandraService.launchCluster(CassandraService.java:109)
>>>> ... 3 more
>>>>
>>>>
>>>>
>>>> On Sat, Jan 29, 2011 at 9:05 PM, Lars George <la...@gmail.com> wrote:
>>>>> Hi Ashish,
>>>>>
>>>>> Looking at the names of your certificate files it seems you have created them up on AWS and downloaded them to your machine - that was how it also had to be done in the past with the previous EC2 scripts. Note though that this is different for Whirr, it requires an OpenSSH key pair as used by your ssh client. Those are stored in $HOME/.ssh/id_rsa (plus the matching id_rsa.pub for the public key). Look into those to see how their format looks like I bet the ones you are using are different.
>>>>>
>>>>> The easiest is really to use your existing SSH key or create one with
>>>>>
>>>>> $ ssh-keygen -t rsa
>>>>>
>>>>> And specify a different directory and point the Whirr .properties to it.
>>>>>
>>>>> Lars
>>>>>
>>>>> On Jan 29, 2011, at 14:13, Ashish <pa...@gmail.com> wrote:
>>>>>
>>>>>> Thanks Andrei for the wonderful explanation :)
>>>>>>
>>>>>> Unfortunately I am still getting some error's
>>>>>>
>>>>>> Here is my config file
>>>>>>
>>>>>> whirr.service-name=cassandra
>>>>>> whirr.cluster-name=mycassandracluster
>>>>>> whirr.instance-templates=3 cassandra
>>>>>> whirr.provider=ec2
>>>>>> whirr.identity=XXXXXXXXXX
>>>>>> whirr.credential=XXXXXXXXXX
>>>>>> whirr.private-key-file=pk-XXXXXXXXXXXX.pem
>>>>>> whirr.public-key-file=rsa-XXXXXXXXXXXX.pem
>>>>>>
>>>>>> Earlier I was using cygwin on windows. I ran this test on Mac today.
>>>>>> Got following error
>>>>>>
>>>>>> ssh, completed: 0/2, errors: 1, rate: 2288ms/op
>>>>>> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
>>>>>> ec2-user@50.16.22.161:22: Error putting path: .ssh/id_rsa.pub
>>>>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>>>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>> at java.lang.Thread.run(Thread.java:680)
>>>>>> Caused by: org.jclouds.ssh.SshException: ec2-user@50.16.22.161:22:
>>>>>> Error putting path: .ssh/id_rsa.pub
>>>>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
>>>>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
>>>>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
>>>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>> ... 3 more
>>>>>> Caused by: 4: java.io.IOException: Pipe closed
>>>>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
>>>>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
>>>>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
>>>>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
>>>>>> ... 7 more
>>>>>> Caused by: java.io.IOException: Pipe closed
>>>>>> at java.io.PipedInputStream.read(PipedInputStream.java:291)
>>>>>> at java.io.PipedInputStream.read(PipedInputStream.java:361)
>>>>>> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
>>>>>> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
>>>>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:474)
>>>>>> ... 10 more
>>>>>>
>>>>>>
>>>>>> From the logs I found this
>>>>>>
>>>>>> 2011-01-29 18:31:54,075 DEBUG [jclouds.compute] (main) >> creating
>>>>>> keyPair region(us-east-1) tag(mycassandracluster)
>>>>>> 2011-01-29 18:31:56,490 DEBUG [jclouds.compute] (main) << created
>>>>>> keyPair(jclouds#mycassandracluster#us-east-1#21)
>>>>>> 2011-01-29 18:31:56,491 DEBUG [jclouds.compute] (main) >> creating
>>>>>> securityGroup region(us-east-1)
>>>>>> name(jclouds#mycassandracluster#us-east-1)
>>>>>>
>>>>>>
>>>>>> seems like the keypair is being created irrespective of the settings
>>>>>> in config file.
>>>>>>
>>>>>> Am I missing something here?
>>>>>>
>>>>>> Appreciate all your help.
>>>>>>
>>>>>> thanks
>>>>>> ashish
>>>>>>
>>>>>> On Sat, Jan 29, 2011 at 5:39 PM, Andrei Savu <sa...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Two questions
>>>>>>>> 1. Is there a way to get the keypair created by whirr?
>>>>>>>
>>>>>>> You can specify the keypair used to access the server in the cluster
>>>>>>> configuration file. By default Whirr uses ~/.ssh/id_rsa and
>>>>>>> ~/.ssh/id_rsa.pub.
>>>>>>>
>>>>>>> This following snippet was taken from a recipe:
>>>>>>>
>>>>>>> # By default use the user system SSH keys. Override them here.
>>>>>>> # whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
>>>>>>> # whirr.public-key-file=${whirr.private-key-file}.pub
>>>>>>>
>>>>>>> You can override the keys specified in the configuration file on the
>>>>>>> command line:
>>>>>>>
>>>>>>> --private-key-file
>>>>>>> --public-key-file
>>>>>>>
>>>>>>> Whirr can only work with SSH RSA keypairs for now.
>>>>>>>
>>>>>>> Note: If you are creating a Hadoop or HBase cluster, Whirr creates
>>>>>>> some utility scripts that you can use to setup a local SOCKS proxy to
>>>>>>> the cluster. Check the configuration guide for more info
>>>>>>> http://incubator.apache.org/whirr/quick-start-guide.html#Run_a_proxy
>>>>>>>
>>>>>>> Also, keep in mind that Whirr does not create a user on the servers
>>>>>>> with the same name as your local user.
>>>>>>>
>>>>>>> Login to EC2 by using: $ ssh -i ssh_rsa_private_key
>>>>>>> ec2-user@public-ec2-server-ip
>>>>>>>
>>>>>>> Login to Rackspace Cloud by using: $ ssh -i ssh_rsa_private_key
>>>>>>> root@public-rackspace-cloud-ip
>>>>>>>
>>>>>>> I hope this clarifies things a bit for you. Let me know if you have
>>>>>>> more questions.
>>>>>>>
>>>>>>>>
>>>>>>>> BTW using whirr for Hadoop has been a pleasure, it rocks :)
>>>>>>>
>>>>>>> Glad you like it!
>>>>>>>
>>>>>>>>
>>>>>>>> thanks
>>>>>>>> ashish
>>>>>>>>
>>>>>>>> On Sat, Jan 29, 2011 at 12:46 AM, Tom White <to...@gmail.com> wrote:
>>>>>>>>> On Fri, Jan 28, 2011 at 12:53 AM, Ashish <pa...@gmail.com> wrote:
>>>>>>>>>> Folks,
>>>>>>>>>>
>>>>>>>>>> I followed the instructions from
>>>>>>>>>> http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
>>>>>>>>>>
>>>>>>>>>> Using whirr-0.2.0-incubating stable release
>>>>>>>>>
>>>>>>>>> I suggest trying with 0.3.0 (out soon, or available from svn now, as
>>>>>>>>> the blog outlines) since the Cassandra code has changed quite a bit
>>>>>>>>> since 0.2.0.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Instances are launched, but at the end it it displays an error while
>>>>>>>>>> connecting it.
>>>>>>>>>>
>>>>>>>>>> Following exception is printed -----------------
>>>>>>>>>>
>>>>>>>>>> Authorizing firewall
>>>>>>>>>> Running configuration script
>>>>>>>>>> Exception in thread "main" java.io.IOException:
>>>>>>>>>> org.jclouds.compute.RunScriptOnNodesException: error runScript on
>>>>>>>>>> filter
>>>>>>>>>> ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
>>>>>>>>>> Execution failures:
>>>>>>>>>>
>>>>>>>>>> 0 error[s]
>>>>>>>>>> Node failures:
>>>>>>>>>>
>>>>>>>>>> 1) SshException on node us-east-1/i-17f3497b:
>>>>>>>>>> org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
>>>>>>>>>> connecting to session.
>>>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
>>>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
>>>>>>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
>>>>>>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
>>>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>>>>>>>>>> at java.util.concurrent.FutureTask.run(Unknown Source)
>>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>>>>>>>>> Source)
>>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>>>>>>>>> at java.lang.Thread.run(Unknown Source)
>>>>>>>>>> Caused by: com.jcraft.jsch.JSchException: Auth fail
>>>>>>>>>> at com.jcraft.jsch.Session.connect(Session.java:452)
>>>>>>>>>> at com.jcraft.jsch.Session.connect(Session.java:150)
>>>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
>>>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
>>>>>>>>>> ... 7 more
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hadoop cluster runs fine with the whirr release.
>>>>>>>>>>
>>>>>>>>>> My configuration file is plan simple, picked from the blog and the
>>>>>>>>>> real value replaced.
>>>>>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> thanks
>>>> ashish
>>>>
>>>> Blog: http://www.ashishpaliwal.com/blog
>>>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>>
>>
>>
>>
>> --
>> thanks
>> ashish
>>
>> Blog: http://www.ashishpaliwal.com/blog
>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Error while running cassandra using whirr
Posted by Ashish <pa...@gmail.com>.
@lars - hbase service is not part of 0.2.0. Let me try with trunk build.
thanks
ashish
On Sat, Jan 29, 2011 at 10:06 PM, Ashish <pa...@gmail.com> wrote:
> On Sat, Jan 29, 2011 at 9:55 PM, Lars George <la...@gmail.com> wrote:
>> Hi Ashish,
>>
>> Cool, and you have stored them in the default location in your home dir? In other words you are sure they get picked up?
>
> yes they are placed in default location. Not sure if they are picked
> up, nothing shows up in logs :(
>
> AMI's run successfully, its later that this problem happens.
>
>>
>> Sometimes I had issues using the default AMIs and instance size, could you try what I am using here
>>
>> https://github.com/larsgeorge/whirr/blob/WHIRR-201/services/hbase/src/test/resources/whirr-hbase-test.properties#L27
>>
>> That is the Alestic AMI with Ubuntu Lucid and at least a medium if not large instance size. If that works you know at least it should work with the defaults.
>
> Let me try with this configuration. Hadoop cluster did ran fine, so am
> suspecting some goofup at my end.
>
> Will revert back with the test result ASAP.
>
>>
>> Lars
>>
>> On Jan 29, 2011, at 17:01, Ashish <pa...@gmail.com> wrote:
>>
>>> @Andrei - using 0.2.0, tried with trunk as well
>>>
>>> @Lars - Bulls eye, I am using the Amazon keys :(
>>> Never crossed my mind that I might need to generate and use the keys
>>>
>>> Since I had a clean machine, I created the ssh keys using the command
>>> u specified
>>>
>>> $ ssh-keygen -t rsa
>>>
>>> Then commented the properties related to keys and ran again. I ran
>>> into another error :(
>>>
>>> Do you think I should move to 0.3.0 and test?
>>>
>>> Sorry for eating up your weekend.
>>>
>>> Traces below
>>> --------
>>>
>>> Configuring template
>>> Starting 3 node(s)
>>> ssh, completed: 0/2, errors: 1, rate: 2802ms/op
>>> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
>>> ec2-user@184.73.14.241:22: Error connecting to sftp.
>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:680)
>>> Caused by: org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22:
>>> Error connecting to sftp.
>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:162)
>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> ... 3 more
>>> Caused by: com.jcraft.jsch.JSchException: java.io.IOException:
>>> inputstream is closed
>>> at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:242)
>>> at com.jcraft.jsch.Channel.connect(Channel.java:200)
>>> at com.jcraft.jsch.Channel.connect(Channel.java:144)
>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:160)
>>> ... 7 more
>>> Caused by: java.io.IOException: inputstream is closed
>>> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
>>> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
>>> at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:211)
>>> ... 10 more
>>> ssh, completed: 0/2, errors: 2, rate: 2803ms/op
>>> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
>>> ec2-user@184.73.14.241:22: Error putting path: runscript
>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:680)
>>> Caused by: org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22:
>>> Error putting path: runscript
>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
>>> at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:94)
>>> at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:53)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> ... 3 more
>>> Caused by: 4: java.io.IOException: inputstream is closed
>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
>>> ... 7 more
>>> Caused by: java.io.IOException: inputstream is closed
>>> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
>>> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
>>> at com.jcraft.jsch.ChannelSftp.checkStatus(ChannelSftp.java:1924)
>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:567)
>>> ... 10 more
>>> ssh, completed: 0/2, errors: 2, rate: 2807ms/op
>>> java.lang.RuntimeException: ssh, completed: 0/2, errors: 2, rate: 2807ms/op
>>> at org.jclouds.concurrent.FutureIterables.awaitCompletion(FutureIterables.java:139)
>>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:245)
>>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:680)
>>> << problem applying options to node(us-east-1/i-831197ef):
>>> java.lang.RuntimeException: error invoking callables on nodes:
>>> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
>>> path: runscript,
>>> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
>>> connecting to sftp.}
>>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
>>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:680)
>>> Exception in thread "main" java.io.IOException:
>>> org.jclouds.compute.RunNodesException: error running 3 nodes
>>> tag(mycassandracluster) location(us-east-1) image(ami-74f0061d)
>>> size(t1.micro) options([groupIds=[], keyPair=null, noKeyPair=false,
>>> placementGroup=null, noPlacementGroup=false, monitoringEnabled=false,
>>> inboundPorts=[22], privateKey=true, publicKey=true, runScript=true,
>>> port:seconds=-1:-1, subnetId=null, metadata/details: false])
>>> Execution failures:
>>>
>>> 0 error[s]
>>> Node failures:
>>>
>>> 1) RuntimeException on node us-east-1/i-831197ef:
>>> java.lang.RuntimeException: error invoking callables on nodes:
>>> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
>>> path: runscript,
>>> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
>>> connecting to sftp.}
>>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
>>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:680)
>>>
>>>
>>> 1 error[s]
>>> at org.apache.whirr.service.cassandra.CassandraService.launchCluster(CassandraService.java:114)
>>> at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:61)
>>> at org.apache.whirr.cli.Main.run(Main.java:65)
>>> at org.apache.whirr.cli.Main.main(Main.java:91)
>>> Caused by: org.jclouds.compute.RunNodesException: error running 3
>>> nodes tag(mycassandracluster) location(us-east-1) image(ami-74f0061d)
>>> size(t1.micro) options([groupIds=[], keyPair=null, noKeyPair=false,
>>> placementGroup=null, noPlacementGroup=false, monitoringEnabled=false,
>>> inboundPorts=[22], privateKey=true, publicKey=true, runScript=true,
>>> port:seconds=-1:-1, subnetId=null, metadata/details: false])
>>> Execution failures:
>>>
>>> 0 error[s]
>>> Node failures:
>>>
>>> 1) RuntimeException on node us-east-1/i-831197ef:
>>> java.lang.RuntimeException: error invoking callables on nodes:
>>> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
>>> path: runscript,
>>> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
>>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
>>> connecting to sftp.}
>>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
>>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:680)
>>>
>>>
>>> 1 error[s]
>>> at org.jclouds.compute.internal.BaseComputeService.runNodesWithTag(BaseComputeService.java:162)
>>> at org.apache.whirr.service.cassandra.CassandraService.launchCluster(CassandraService.java:109)
>>> ... 3 more
>>>
>>>
>>>
>>> On Sat, Jan 29, 2011 at 9:05 PM, Lars George <la...@gmail.com> wrote:
>>>> Hi Ashish,
>>>>
>>>> Looking at the names of your certificate files it seems you have created them up on AWS and downloaded them to your machine - that was how it also had to be done in the past with the previous EC2 scripts. Note though that this is different for Whirr, it requires an OpenSSH key pair as used by your ssh client. Those are stored in $HOME/.ssh/id_rsa (plus the matching id_rsa.pub for the public key). Look into those to see how their format looks like I bet the ones you are using are different.
>>>>
>>>> The easiest is really to use your existing SSH key or create one with
>>>>
>>>> $ ssh-keygen -t rsa
>>>>
>>>> And specify a different directory and point the Whirr .properties to it.
>>>>
>>>> Lars
>>>>
>>>> On Jan 29, 2011, at 14:13, Ashish <pa...@gmail.com> wrote:
>>>>
>>>>> Thanks Andrei for the wonderful explanation :)
>>>>>
>>>>> Unfortunately I am still getting some error's
>>>>>
>>>>> Here is my config file
>>>>>
>>>>> whirr.service-name=cassandra
>>>>> whirr.cluster-name=mycassandracluster
>>>>> whirr.instance-templates=3 cassandra
>>>>> whirr.provider=ec2
>>>>> whirr.identity=XXXXXXXXXX
>>>>> whirr.credential=XXXXXXXXXX
>>>>> whirr.private-key-file=pk-XXXXXXXXXXXX.pem
>>>>> whirr.public-key-file=rsa-XXXXXXXXXXXX.pem
>>>>>
>>>>> Earlier I was using cygwin on windows. I ran this test on Mac today.
>>>>> Got following error
>>>>>
>>>>> ssh, completed: 0/2, errors: 1, rate: 2288ms/op
>>>>> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
>>>>> ec2-user@50.16.22.161:22: Error putting path: .ssh/id_rsa.pub
>>>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>> at java.lang.Thread.run(Thread.java:680)
>>>>> Caused by: org.jclouds.ssh.SshException: ec2-user@50.16.22.161:22:
>>>>> Error putting path: .ssh/id_rsa.pub
>>>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
>>>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
>>>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
>>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>> ... 3 more
>>>>> Caused by: 4: java.io.IOException: Pipe closed
>>>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
>>>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
>>>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
>>>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
>>>>> ... 7 more
>>>>> Caused by: java.io.IOException: Pipe closed
>>>>> at java.io.PipedInputStream.read(PipedInputStream.java:291)
>>>>> at java.io.PipedInputStream.read(PipedInputStream.java:361)
>>>>> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
>>>>> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
>>>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:474)
>>>>> ... 10 more
>>>>>
>>>>>
>>>>> From the logs I found this
>>>>>
>>>>> 2011-01-29 18:31:54,075 DEBUG [jclouds.compute] (main) >> creating
>>>>> keyPair region(us-east-1) tag(mycassandracluster)
>>>>> 2011-01-29 18:31:56,490 DEBUG [jclouds.compute] (main) << created
>>>>> keyPair(jclouds#mycassandracluster#us-east-1#21)
>>>>> 2011-01-29 18:31:56,491 DEBUG [jclouds.compute] (main) >> creating
>>>>> securityGroup region(us-east-1)
>>>>> name(jclouds#mycassandracluster#us-east-1)
>>>>>
>>>>>
>>>>> seems like the keypair is being created irrespective of the settings
>>>>> in config file.
>>>>>
>>>>> Am I missing something here?
>>>>>
>>>>> Appreciate all your help.
>>>>>
>>>>> thanks
>>>>> ashish
>>>>>
>>>>> On Sat, Jan 29, 2011 at 5:39 PM, Andrei Savu <sa...@gmail.com> wrote:
>>>>>>>
>>>>>>> Two questions
>>>>>>> 1. Is there a way to get the keypair created by whirr?
>>>>>>
>>>>>> You can specify the keypair used to access the server in the cluster
>>>>>> configuration file. By default Whirr uses ~/.ssh/id_rsa and
>>>>>> ~/.ssh/id_rsa.pub.
>>>>>>
>>>>>> This following snippet was taken from a recipe:
>>>>>>
>>>>>> # By default use the user system SSH keys. Override them here.
>>>>>> # whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
>>>>>> # whirr.public-key-file=${whirr.private-key-file}.pub
>>>>>>
>>>>>> You can override the keys specified in the configuration file on the
>>>>>> command line:
>>>>>>
>>>>>> --private-key-file
>>>>>> --public-key-file
>>>>>>
>>>>>> Whirr can only work with SSH RSA keypairs for now.
>>>>>>
>>>>>> Note: If you are creating a Hadoop or HBase cluster, Whirr creates
>>>>>> some utility scripts that you can use to setup a local SOCKS proxy to
>>>>>> the cluster. Check the configuration guide for more info
>>>>>> http://incubator.apache.org/whirr/quick-start-guide.html#Run_a_proxy
>>>>>>
>>>>>> Also, keep in mind that Whirr does not create a user on the servers
>>>>>> with the same name as your local user.
>>>>>>
>>>>>> Login to EC2 by using: $ ssh -i ssh_rsa_private_key
>>>>>> ec2-user@public-ec2-server-ip
>>>>>>
>>>>>> Login to Rackspace Cloud by using: $ ssh -i ssh_rsa_private_key
>>>>>> root@public-rackspace-cloud-ip
>>>>>>
>>>>>> I hope this clarifies things a bit for you. Let me know if you have
>>>>>> more questions.
>>>>>>
>>>>>>>
>>>>>>> BTW using whirr for Hadoop has been a pleasure, it rocks :)
>>>>>>
>>>>>> Glad you like it!
>>>>>>
>>>>>>>
>>>>>>> thanks
>>>>>>> ashish
>>>>>>>
>>>>>>> On Sat, Jan 29, 2011 at 12:46 AM, Tom White <to...@gmail.com> wrote:
>>>>>>>> On Fri, Jan 28, 2011 at 12:53 AM, Ashish <pa...@gmail.com> wrote:
>>>>>>>>> Folks,
>>>>>>>>>
>>>>>>>>> I followed the instructions from
>>>>>>>>> http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
>>>>>>>>>
>>>>>>>>> Using whirr-0.2.0-incubating stable release
>>>>>>>>
>>>>>>>> I suggest trying with 0.3.0 (out soon, or available from svn now, as
>>>>>>>> the blog outlines) since the Cassandra code has changed quite a bit
>>>>>>>> since 0.2.0.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Instances are launched, but at the end it it displays an error while
>>>>>>>>> connecting it.
>>>>>>>>>
>>>>>>>>> Following exception is printed -----------------
>>>>>>>>>
>>>>>>>>> Authorizing firewall
>>>>>>>>> Running configuration script
>>>>>>>>> Exception in thread "main" java.io.IOException:
>>>>>>>>> org.jclouds.compute.RunScriptOnNodesException: error runScript on
>>>>>>>>> filter
>>>>>>>>> ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
>>>>>>>>> Execution failures:
>>>>>>>>>
>>>>>>>>> 0 error[s]
>>>>>>>>> Node failures:
>>>>>>>>>
>>>>>>>>> 1) SshException on node us-east-1/i-17f3497b:
>>>>>>>>> org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
>>>>>>>>> connecting to session.
>>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
>>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
>>>>>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
>>>>>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
>>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>>>>>>>>> at java.util.concurrent.FutureTask.run(Unknown Source)
>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>>>>>>>> Source)
>>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>>>>>>>> at java.lang.Thread.run(Unknown Source)
>>>>>>>>> Caused by: com.jcraft.jsch.JSchException: Auth fail
>>>>>>>>> at com.jcraft.jsch.Session.connect(Session.java:452)
>>>>>>>>> at com.jcraft.jsch.Session.connect(Session.java:150)
>>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
>>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
>>>>>>>>> ... 7 more
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hadoop cluster runs fine with the whirr release.
>>>>>>>>>
>>>>>>>>> My configuration file is plan simple, picked from the blog and the
>>>>>>>>> real value replaced.
>>>>>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> thanks
>>> ashish
>>>
>>> Blog: http://www.ashishpaliwal.com/blog
>>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Error while running cassandra using whirr
Posted by Ashish <pa...@gmail.com>.
On Sat, Jan 29, 2011 at 9:55 PM, Lars George <la...@gmail.com> wrote:
> Hi Ashish,
>
> Cool, and you have stored them in the default location in your home dir? In other words you are sure they get picked up?
yes they are placed in default location. Not sure if they are picked
up, nothing shows up in logs :(
AMI's run successfully, its later that this problem happens.
>
> Sometimes I had issues using the default AMIs and instance size, could you try what I am using here
>
> https://github.com/larsgeorge/whirr/blob/WHIRR-201/services/hbase/src/test/resources/whirr-hbase-test.properties#L27
>
> That is the Alestic AMI with Ubuntu Lucid and at least a medium if not large instance size. If that works you know at least it should work with the defaults.
Let me try with this configuration. Hadoop cluster did ran fine, so am
suspecting some goofup at my end.
Will revert back with the test result ASAP.
>
> Lars
>
> On Jan 29, 2011, at 17:01, Ashish <pa...@gmail.com> wrote:
>
>> @Andrei - using 0.2.0, tried with trunk as well
>>
>> @Lars - Bulls eye, I am using the Amazon keys :(
>> Never crossed my mind that I might need to generate and use the keys
>>
>> Since I had a clean machine, I created the ssh keys using the command
>> u specified
>>
>> $ ssh-keygen -t rsa
>>
>> Then commented the properties related to keys and ran again. I ran
>> into another error :(
>>
>> Do you think I should move to 0.3.0 and test?
>>
>> Sorry for eating up your weekend.
>>
>> Traces below
>> --------
>>
>> Configuring template
>> Starting 3 node(s)
>> ssh, completed: 0/2, errors: 1, rate: 2802ms/op
>> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
>> ec2-user@184.73.14.241:22: Error connecting to sftp.
>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:680)
>> Caused by: org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22:
>> Error connecting to sftp.
>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:162)
>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> ... 3 more
>> Caused by: com.jcraft.jsch.JSchException: java.io.IOException:
>> inputstream is closed
>> at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:242)
>> at com.jcraft.jsch.Channel.connect(Channel.java:200)
>> at com.jcraft.jsch.Channel.connect(Channel.java:144)
>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:160)
>> ... 7 more
>> Caused by: java.io.IOException: inputstream is closed
>> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
>> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
>> at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:211)
>> ... 10 more
>> ssh, completed: 0/2, errors: 2, rate: 2803ms/op
>> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
>> ec2-user@184.73.14.241:22: Error putting path: runscript
>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:680)
>> Caused by: org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22:
>> Error putting path: runscript
>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
>> at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:94)
>> at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:53)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> ... 3 more
>> Caused by: 4: java.io.IOException: inputstream is closed
>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
>> ... 7 more
>> Caused by: java.io.IOException: inputstream is closed
>> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
>> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
>> at com.jcraft.jsch.ChannelSftp.checkStatus(ChannelSftp.java:1924)
>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:567)
>> ... 10 more
>> ssh, completed: 0/2, errors: 2, rate: 2807ms/op
>> java.lang.RuntimeException: ssh, completed: 0/2, errors: 2, rate: 2807ms/op
>> at org.jclouds.concurrent.FutureIterables.awaitCompletion(FutureIterables.java:139)
>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:245)
>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:680)
>> << problem applying options to node(us-east-1/i-831197ef):
>> java.lang.RuntimeException: error invoking callables on nodes:
>> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
>> path: runscript,
>> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
>> connecting to sftp.}
>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:680)
>> Exception in thread "main" java.io.IOException:
>> org.jclouds.compute.RunNodesException: error running 3 nodes
>> tag(mycassandracluster) location(us-east-1) image(ami-74f0061d)
>> size(t1.micro) options([groupIds=[], keyPair=null, noKeyPair=false,
>> placementGroup=null, noPlacementGroup=false, monitoringEnabled=false,
>> inboundPorts=[22], privateKey=true, publicKey=true, runScript=true,
>> port:seconds=-1:-1, subnetId=null, metadata/details: false])
>> Execution failures:
>>
>> 0 error[s]
>> Node failures:
>>
>> 1) RuntimeException on node us-east-1/i-831197ef:
>> java.lang.RuntimeException: error invoking callables on nodes:
>> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
>> path: runscript,
>> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
>> connecting to sftp.}
>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:680)
>>
>>
>> 1 error[s]
>> at org.apache.whirr.service.cassandra.CassandraService.launchCluster(CassandraService.java:114)
>> at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:61)
>> at org.apache.whirr.cli.Main.run(Main.java:65)
>> at org.apache.whirr.cli.Main.main(Main.java:91)
>> Caused by: org.jclouds.compute.RunNodesException: error running 3
>> nodes tag(mycassandracluster) location(us-east-1) image(ami-74f0061d)
>> size(t1.micro) options([groupIds=[], keyPair=null, noKeyPair=false,
>> placementGroup=null, noPlacementGroup=false, monitoringEnabled=false,
>> inboundPorts=[22], privateKey=true, publicKey=true, runScript=true,
>> port:seconds=-1:-1, subnetId=null, metadata/details: false])
>> Execution failures:
>>
>> 0 error[s]
>> Node failures:
>>
>> 1) RuntimeException on node us-east-1/i-831197ef:
>> java.lang.RuntimeException: error invoking callables on nodes:
>> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
>> path: runscript,
>> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
>> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
>> connecting to sftp.}
>> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
>> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
>> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
>> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
>> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:680)
>>
>>
>> 1 error[s]
>> at org.jclouds.compute.internal.BaseComputeService.runNodesWithTag(BaseComputeService.java:162)
>> at org.apache.whirr.service.cassandra.CassandraService.launchCluster(CassandraService.java:109)
>> ... 3 more
>>
>>
>>
>> On Sat, Jan 29, 2011 at 9:05 PM, Lars George <la...@gmail.com> wrote:
>>> Hi Ashish,
>>>
>>> Looking at the names of your certificate files it seems you have created them up on AWS and downloaded them to your machine - that was how it also had to be done in the past with the previous EC2 scripts. Note though that this is different for Whirr, it requires an OpenSSH key pair as used by your ssh client. Those are stored in $HOME/.ssh/id_rsa (plus the matching id_rsa.pub for the public key). Look into those to see how their format looks like I bet the ones you are using are different.
>>>
>>> The easiest is really to use your existing SSH key or create one with
>>>
>>> $ ssh-keygen -t rsa
>>>
>>> And specify a different directory and point the Whirr .properties to it.
>>>
>>> Lars
>>>
>>> On Jan 29, 2011, at 14:13, Ashish <pa...@gmail.com> wrote:
>>>
>>>> Thanks Andrei for the wonderful explanation :)
>>>>
>>>> Unfortunately I am still getting some error's
>>>>
>>>> Here is my config file
>>>>
>>>> whirr.service-name=cassandra
>>>> whirr.cluster-name=mycassandracluster
>>>> whirr.instance-templates=3 cassandra
>>>> whirr.provider=ec2
>>>> whirr.identity=XXXXXXXXXX
>>>> whirr.credential=XXXXXXXXXX
>>>> whirr.private-key-file=pk-XXXXXXXXXXXX.pem
>>>> whirr.public-key-file=rsa-XXXXXXXXXXXX.pem
>>>>
>>>> Earlier I was using cygwin on windows. I ran this test on Mac today.
>>>> Got following error
>>>>
>>>> ssh, completed: 0/2, errors: 1, rate: 2288ms/op
>>>> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
>>>> ec2-user@50.16.22.161:22: Error putting path: .ssh/id_rsa.pub
>>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:680)
>>>> Caused by: org.jclouds.ssh.SshException: ec2-user@50.16.22.161:22:
>>>> Error putting path: .ssh/id_rsa.pub
>>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
>>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
>>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> ... 3 more
>>>> Caused by: 4: java.io.IOException: Pipe closed
>>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
>>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
>>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
>>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
>>>> ... 7 more
>>>> Caused by: java.io.IOException: Pipe closed
>>>> at java.io.PipedInputStream.read(PipedInputStream.java:291)
>>>> at java.io.PipedInputStream.read(PipedInputStream.java:361)
>>>> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
>>>> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
>>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:474)
>>>> ... 10 more
>>>>
>>>>
>>>> From the logs I found this
>>>>
>>>> 2011-01-29 18:31:54,075 DEBUG [jclouds.compute] (main) >> creating
>>>> keyPair region(us-east-1) tag(mycassandracluster)
>>>> 2011-01-29 18:31:56,490 DEBUG [jclouds.compute] (main) << created
>>>> keyPair(jclouds#mycassandracluster#us-east-1#21)
>>>> 2011-01-29 18:31:56,491 DEBUG [jclouds.compute] (main) >> creating
>>>> securityGroup region(us-east-1)
>>>> name(jclouds#mycassandracluster#us-east-1)
>>>>
>>>>
>>>> seems like the keypair is being created irrespective of the settings
>>>> in config file.
>>>>
>>>> Am I missing something here?
>>>>
>>>> Appreciate all your help.
>>>>
>>>> thanks
>>>> ashish
>>>>
>>>> On Sat, Jan 29, 2011 at 5:39 PM, Andrei Savu <sa...@gmail.com> wrote:
>>>>>>
>>>>>> Two questions
>>>>>> 1. Is there a way to get the keypair created by whirr?
>>>>>
>>>>> You can specify the keypair used to access the server in the cluster
>>>>> configuration file. By default Whirr uses ~/.ssh/id_rsa and
>>>>> ~/.ssh/id_rsa.pub.
>>>>>
>>>>> This following snippet was taken from a recipe:
>>>>>
>>>>> # By default use the user system SSH keys. Override them here.
>>>>> # whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
>>>>> # whirr.public-key-file=${whirr.private-key-file}.pub
>>>>>
>>>>> You can override the keys specified in the configuration file on the
>>>>> command line:
>>>>>
>>>>> --private-key-file
>>>>> --public-key-file
>>>>>
>>>>> Whirr can only work with SSH RSA keypairs for now.
>>>>>
>>>>> Note: If you are creating a Hadoop or HBase cluster, Whirr creates
>>>>> some utility scripts that you can use to setup a local SOCKS proxy to
>>>>> the cluster. Check the configuration guide for more info
>>>>> http://incubator.apache.org/whirr/quick-start-guide.html#Run_a_proxy
>>>>>
>>>>> Also, keep in mind that Whirr does not create a user on the servers
>>>>> with the same name as your local user.
>>>>>
>>>>> Login to EC2 by using: $ ssh -i ssh_rsa_private_key
>>>>> ec2-user@public-ec2-server-ip
>>>>>
>>>>> Login to Rackspace Cloud by using: $ ssh -i ssh_rsa_private_key
>>>>> root@public-rackspace-cloud-ip
>>>>>
>>>>> I hope this clarifies things a bit for you. Let me know if you have
>>>>> more questions.
>>>>>
>>>>>>
>>>>>> BTW using whirr for Hadoop has been a pleasure, it rocks :)
>>>>>
>>>>> Glad you like it!
>>>>>
>>>>>>
>>>>>> thanks
>>>>>> ashish
>>>>>>
>>>>>> On Sat, Jan 29, 2011 at 12:46 AM, Tom White <to...@gmail.com> wrote:
>>>>>>> On Fri, Jan 28, 2011 at 12:53 AM, Ashish <pa...@gmail.com> wrote:
>>>>>>>> Folks,
>>>>>>>>
>>>>>>>> I followed the instructions from
>>>>>>>> http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
>>>>>>>>
>>>>>>>> Using whirr-0.2.0-incubating stable release
>>>>>>>
>>>>>>> I suggest trying with 0.3.0 (out soon, or available from svn now, as
>>>>>>> the blog outlines) since the Cassandra code has changed quite a bit
>>>>>>> since 0.2.0.
>>>>>>>
>>>>>>>>
>>>>>>>> Instances are launched, but at the end it it displays an error while
>>>>>>>> connecting it.
>>>>>>>>
>>>>>>>> Following exception is printed -----------------
>>>>>>>>
>>>>>>>> Authorizing firewall
>>>>>>>> Running configuration script
>>>>>>>> Exception in thread "main" java.io.IOException:
>>>>>>>> org.jclouds.compute.RunScriptOnNodesException: error runScript on
>>>>>>>> filter
>>>>>>>> ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
>>>>>>>> Execution failures:
>>>>>>>>
>>>>>>>> 0 error[s]
>>>>>>>> Node failures:
>>>>>>>>
>>>>>>>> 1) SshException on node us-east-1/i-17f3497b:
>>>>>>>> org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
>>>>>>>> connecting to session.
>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
>>>>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
>>>>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>>>>>>>> at java.util.concurrent.FutureTask.run(Unknown Source)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>>>>>>> Source)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>>>>>>> at java.lang.Thread.run(Unknown Source)
>>>>>>>> Caused by: com.jcraft.jsch.JSchException: Auth fail
>>>>>>>> at com.jcraft.jsch.Session.connect(Session.java:452)
>>>>>>>> at com.jcraft.jsch.Session.connect(Session.java:150)
>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
>>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
>>>>>>>> ... 7 more
>>>>>>>>
>>>>>>>>
>>>>>>>> Hadoop cluster runs fine with the whirr release.
>>>>>>>>
>>>>>>>> My configuration file is plan simple, picked from the blog and the
>>>>>>>> real value replaced.
>>>>>>>>
>>>
>>
>>
>>
>> --
>> thanks
>> ashish
>>
>> Blog: http://www.ashishpaliwal.com/blog
>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Error while running cassandra using whirr
Posted by Lars George <la...@gmail.com>.
Hi Ashish,
Cool, and you have stored them in the default location in your home dir? In other words you are sure they get picked up?
Sometimes I had issues using the default AMIs and instance size, could you try what I am using here
https://github.com/larsgeorge/whirr/blob/WHIRR-201/services/hbase/src/test/resources/whirr-hbase-test.properties#L27
That is the Alestic AMI with Ubuntu Lucid and at least a medium if not large instance size. If that works you know at least it should work with the defaults.
Lars
On Jan 29, 2011, at 17:01, Ashish <pa...@gmail.com> wrote:
> @Andrei - using 0.2.0, tried with trunk as well
>
> @Lars - Bulls eye, I am using the Amazon keys :(
> Never crossed my mind that I might need to generate and use the keys
>
> Since I had a clean machine, I created the ssh keys using the command
> u specified
>
> $ ssh-keygen -t rsa
>
> Then commented the properties related to keys and ran again. I ran
> into another error :(
>
> Do you think I should move to 0.3.0 and test?
>
> Sorry for eating up your weekend.
>
> Traces below
> --------
>
> Configuring template
> Starting 3 node(s)
> ssh, completed: 0/2, errors: 1, rate: 2802ms/op
> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
> ec2-user@184.73.14.241:22: Error connecting to sftp.
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22:
> Error connecting to sftp.
> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:162)
> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> Caused by: com.jcraft.jsch.JSchException: java.io.IOException:
> inputstream is closed
> at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:242)
> at com.jcraft.jsch.Channel.connect(Channel.java:200)
> at com.jcraft.jsch.Channel.connect(Channel.java:144)
> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:160)
> ... 7 more
> Caused by: java.io.IOException: inputstream is closed
> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
> at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:211)
> ... 10 more
> ssh, completed: 0/2, errors: 2, rate: 2803ms/op
> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
> ec2-user@184.73.14.241:22: Error putting path: runscript
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22:
> Error putting path: runscript
> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
> at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:94)
> at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:53)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> Caused by: 4: java.io.IOException: inputstream is closed
> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
> ... 7 more
> Caused by: java.io.IOException: inputstream is closed
> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
> at com.jcraft.jsch.ChannelSftp.checkStatus(ChannelSftp.java:1924)
> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:567)
> ... 10 more
> ssh, completed: 0/2, errors: 2, rate: 2807ms/op
> java.lang.RuntimeException: ssh, completed: 0/2, errors: 2, rate: 2807ms/op
> at org.jclouds.concurrent.FutureIterables.awaitCompletion(FutureIterables.java:139)
> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:245)
> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> << problem applying options to node(us-east-1/i-831197ef):
> java.lang.RuntimeException: error invoking callables on nodes:
> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
> path: runscript,
> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
> connecting to sftp.}
> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Exception in thread "main" java.io.IOException:
> org.jclouds.compute.RunNodesException: error running 3 nodes
> tag(mycassandracluster) location(us-east-1) image(ami-74f0061d)
> size(t1.micro) options([groupIds=[], keyPair=null, noKeyPair=false,
> placementGroup=null, noPlacementGroup=false, monitoringEnabled=false,
> inboundPorts=[22], privateKey=true, publicKey=true, runScript=true,
> port:seconds=-1:-1, subnetId=null, metadata/details: false])
> Execution failures:
>
> 0 error[s]
> Node failures:
>
> 1) RuntimeException on node us-east-1/i-831197ef:
> java.lang.RuntimeException: error invoking callables on nodes:
> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
> path: runscript,
> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
> connecting to sftp.}
> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
>
>
> 1 error[s]
> at org.apache.whirr.service.cassandra.CassandraService.launchCluster(CassandraService.java:114)
> at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:61)
> at org.apache.whirr.cli.Main.run(Main.java:65)
> at org.apache.whirr.cli.Main.main(Main.java:91)
> Caused by: org.jclouds.compute.RunNodesException: error running 3
> nodes tag(mycassandracluster) location(us-east-1) image(ami-74f0061d)
> size(t1.micro) options([groupIds=[], keyPair=null, noKeyPair=false,
> placementGroup=null, noPlacementGroup=false, monitoringEnabled=false,
> inboundPorts=[22], privateKey=true, publicKey=true, runScript=true,
> port:seconds=-1:-1, subnetId=null, metadata/details: false])
> Execution failures:
>
> 0 error[s]
> Node failures:
>
> 1) RuntimeException on node us-east-1/i-831197ef:
> java.lang.RuntimeException: error invoking callables on nodes:
> {org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
> path: runscript,
> org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
> org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
> connecting to sftp.}
> at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
> at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
> at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
> at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
> at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
>
>
> 1 error[s]
> at org.jclouds.compute.internal.BaseComputeService.runNodesWithTag(BaseComputeService.java:162)
> at org.apache.whirr.service.cassandra.CassandraService.launchCluster(CassandraService.java:109)
> ... 3 more
>
>
>
> On Sat, Jan 29, 2011 at 9:05 PM, Lars George <la...@gmail.com> wrote:
>> Hi Ashish,
>>
>> Looking at the names of your certificate files it seems you have created them up on AWS and downloaded them to your machine - that was how it also had to be done in the past with the previous EC2 scripts. Note though that this is different for Whirr, it requires an OpenSSH key pair as used by your ssh client. Those are stored in $HOME/.ssh/id_rsa (plus the matching id_rsa.pub for the public key). Look into those to see how their format looks like I bet the ones you are using are different.
>>
>> The easiest is really to use your existing SSH key or create one with
>>
>> $ ssh-keygen -t rsa
>>
>> And specify a different directory and point the Whirr .properties to it.
>>
>> Lars
>>
>> On Jan 29, 2011, at 14:13, Ashish <pa...@gmail.com> wrote:
>>
>>> Thanks Andrei for the wonderful explanation :)
>>>
>>> Unfortunately I am still getting some error's
>>>
>>> Here is my config file
>>>
>>> whirr.service-name=cassandra
>>> whirr.cluster-name=mycassandracluster
>>> whirr.instance-templates=3 cassandra
>>> whirr.provider=ec2
>>> whirr.identity=XXXXXXXXXX
>>> whirr.credential=XXXXXXXXXX
>>> whirr.private-key-file=pk-XXXXXXXXXXXX.pem
>>> whirr.public-key-file=rsa-XXXXXXXXXXXX.pem
>>>
>>> Earlier I was using cygwin on windows. I ran this test on Mac today.
>>> Got following error
>>>
>>> ssh, completed: 0/2, errors: 1, rate: 2288ms/op
>>> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
>>> ec2-user@50.16.22.161:22: Error putting path: .ssh/id_rsa.pub
>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:680)
>>> Caused by: org.jclouds.ssh.SshException: ec2-user@50.16.22.161:22:
>>> Error putting path: .ssh/id_rsa.pub
>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
>>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> ... 3 more
>>> Caused by: 4: java.io.IOException: Pipe closed
>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
>>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
>>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
>>> ... 7 more
>>> Caused by: java.io.IOException: Pipe closed
>>> at java.io.PipedInputStream.read(PipedInputStream.java:291)
>>> at java.io.PipedInputStream.read(PipedInputStream.java:361)
>>> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
>>> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
>>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:474)
>>> ... 10 more
>>>
>>>
>>> From the logs I found this
>>>
>>> 2011-01-29 18:31:54,075 DEBUG [jclouds.compute] (main) >> creating
>>> keyPair region(us-east-1) tag(mycassandracluster)
>>> 2011-01-29 18:31:56,490 DEBUG [jclouds.compute] (main) << created
>>> keyPair(jclouds#mycassandracluster#us-east-1#21)
>>> 2011-01-29 18:31:56,491 DEBUG [jclouds.compute] (main) >> creating
>>> securityGroup region(us-east-1)
>>> name(jclouds#mycassandracluster#us-east-1)
>>>
>>>
>>> seems like the keypair is being created irrespective of the settings
>>> in config file.
>>>
>>> Am I missing something here?
>>>
>>> Appreciate all your help.
>>>
>>> thanks
>>> ashish
>>>
>>> On Sat, Jan 29, 2011 at 5:39 PM, Andrei Savu <sa...@gmail.com> wrote:
>>>>>
>>>>> Two questions
>>>>> 1. Is there a way to get the keypair created by whirr?
>>>>
>>>> You can specify the keypair used to access the server in the cluster
>>>> configuration file. By default Whirr uses ~/.ssh/id_rsa and
>>>> ~/.ssh/id_rsa.pub.
>>>>
>>>> This following snippet was taken from a recipe:
>>>>
>>>> # By default use the user system SSH keys. Override them here.
>>>> # whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
>>>> # whirr.public-key-file=${whirr.private-key-file}.pub
>>>>
>>>> You can override the keys specified in the configuration file on the
>>>> command line:
>>>>
>>>> --private-key-file
>>>> --public-key-file
>>>>
>>>> Whirr can only work with SSH RSA keypairs for now.
>>>>
>>>> Note: If you are creating a Hadoop or HBase cluster, Whirr creates
>>>> some utility scripts that you can use to setup a local SOCKS proxy to
>>>> the cluster. Check the configuration guide for more info
>>>> http://incubator.apache.org/whirr/quick-start-guide.html#Run_a_proxy
>>>>
>>>> Also, keep in mind that Whirr does not create a user on the servers
>>>> with the same name as your local user.
>>>>
>>>> Login to EC2 by using: $ ssh -i ssh_rsa_private_key
>>>> ec2-user@public-ec2-server-ip
>>>>
>>>> Login to Rackspace Cloud by using: $ ssh -i ssh_rsa_private_key
>>>> root@public-rackspace-cloud-ip
>>>>
>>>> I hope this clarifies things a bit for you. Let me know if you have
>>>> more questions.
>>>>
>>>>>
>>>>> BTW using whirr for Hadoop has been a pleasure, it rocks :)
>>>>
>>>> Glad you like it!
>>>>
>>>>>
>>>>> thanks
>>>>> ashish
>>>>>
>>>>> On Sat, Jan 29, 2011 at 12:46 AM, Tom White <to...@gmail.com> wrote:
>>>>>> On Fri, Jan 28, 2011 at 12:53 AM, Ashish <pa...@gmail.com> wrote:
>>>>>>> Folks,
>>>>>>>
>>>>>>> I followed the instructions from
>>>>>>> http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
>>>>>>>
>>>>>>> Using whirr-0.2.0-incubating stable release
>>>>>>
>>>>>> I suggest trying with 0.3.0 (out soon, or available from svn now, as
>>>>>> the blog outlines) since the Cassandra code has changed quite a bit
>>>>>> since 0.2.0.
>>>>>>
>>>>>>>
>>>>>>> Instances are launched, but at the end it it displays an error while
>>>>>>> connecting it.
>>>>>>>
>>>>>>> Following exception is printed -----------------
>>>>>>>
>>>>>>> Authorizing firewall
>>>>>>> Running configuration script
>>>>>>> Exception in thread "main" java.io.IOException:
>>>>>>> org.jclouds.compute.RunScriptOnNodesException: error runScript on
>>>>>>> filter
>>>>>>> ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
>>>>>>> Execution failures:
>>>>>>>
>>>>>>> 0 error[s]
>>>>>>> Node failures:
>>>>>>>
>>>>>>> 1) SshException on node us-east-1/i-17f3497b:
>>>>>>> org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
>>>>>>> connecting to session.
>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
>>>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
>>>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>>>>>>> at java.util.concurrent.FutureTask.run(Unknown Source)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>>>>>> Source)
>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>>>>>> at java.lang.Thread.run(Unknown Source)
>>>>>>> Caused by: com.jcraft.jsch.JSchException: Auth fail
>>>>>>> at com.jcraft.jsch.Session.connect(Session.java:452)
>>>>>>> at com.jcraft.jsch.Session.connect(Session.java:150)
>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
>>>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
>>>>>>> ... 7 more
>>>>>>>
>>>>>>>
>>>>>>> Hadoop cluster runs fine with the whirr release.
>>>>>>>
>>>>>>> My configuration file is plan simple, picked from the blog and the
>>>>>>> real value replaced.
>>>>>>>
>>
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Error while running cassandra using whirr
Posted by Ashish <pa...@gmail.com>.
@Andrei - using 0.2.0, tried with trunk as well
@Lars - Bulls eye, I am using the Amazon keys :(
Never crossed my mind that I might need to generate and use the keys
Since I had a clean machine, I created the ssh keys using the command
u specified
$ ssh-keygen -t rsa
Then commented the properties related to keys and ran again. I ran
into another error :(
Do you think I should move to 0.3.0 and test?
Sorry for eating up your weekend.
Traces below
--------
Configuring template
Starting 3 node(s)
ssh, completed: 0/2, errors: 1, rate: 2802ms/op
java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
ec2-user@184.73.14.241:22: Error connecting to sftp.
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22:
Error connecting to sftp.
at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:162)
at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: com.jcraft.jsch.JSchException: java.io.IOException:
inputstream is closed
at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:242)
at com.jcraft.jsch.Channel.connect(Channel.java:200)
at com.jcraft.jsch.Channel.connect(Channel.java:144)
at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:160)
... 7 more
Caused by: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:211)
... 10 more
ssh, completed: 0/2, errors: 2, rate: 2803ms/op
java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
ec2-user@184.73.14.241:22: Error putting path: runscript
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22:
Error putting path: runscript
at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:94)
at org.jclouds.compute.callables.RunScriptOnNode.call(RunScriptOnNode.java:53)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: 4: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
... 7 more
Caused by: java.io.IOException: inputstream is closed
at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2327)
at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
at com.jcraft.jsch.ChannelSftp.checkStatus(ChannelSftp.java:1924)
at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:567)
... 10 more
ssh, completed: 0/2, errors: 2, rate: 2807ms/op
java.lang.RuntimeException: ssh, completed: 0/2, errors: 2, rate: 2807ms/op
at org.jclouds.concurrent.FutureIterables.awaitCompletion(FutureIterables.java:139)
at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:245)
at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
<< problem applying options to node(us-east-1/i-831197ef):
java.lang.RuntimeException: error invoking callables on nodes:
{org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
path: runscript,
org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
connecting to sftp.}
at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "main" java.io.IOException:
org.jclouds.compute.RunNodesException: error running 3 nodes
tag(mycassandracluster) location(us-east-1) image(ami-74f0061d)
size(t1.micro) options([groupIds=[], keyPair=null, noKeyPair=false,
placementGroup=null, noPlacementGroup=false, monitoringEnabled=false,
inboundPorts=[22], privateKey=true, publicKey=true, runScript=true,
port:seconds=-1:-1, subnetId=null, metadata/details: false])
Execution failures:
0 error[s]
Node failures:
1) RuntimeException on node us-east-1/i-831197ef:
java.lang.RuntimeException: error invoking callables on nodes:
{org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
path: runscript,
org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
connecting to sftp.}
at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
1 error[s]
at org.apache.whirr.service.cassandra.CassandraService.launchCluster(CassandraService.java:114)
at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:61)
at org.apache.whirr.cli.Main.run(Main.java:65)
at org.apache.whirr.cli.Main.main(Main.java:91)
Caused by: org.jclouds.compute.RunNodesException: error running 3
nodes tag(mycassandracluster) location(us-east-1) image(ami-74f0061d)
size(t1.micro) options([groupIds=[], keyPair=null, noKeyPair=false,
placementGroup=null, noPlacementGroup=false, monitoringEnabled=false,
inboundPorts=[22], privateKey=true, publicKey=true, runScript=true,
port:seconds=-1:-1, subnetId=null, metadata/details: false])
Execution failures:
0 error[s]
Node failures:
1) RuntimeException on node us-east-1/i-831197ef:
java.lang.RuntimeException: error invoking callables on nodes:
{org.jclouds.compute.callables.RunScriptOnNode@244e5056=java.util.concurrent.ExecutionException:
org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error putting
path: runscript,
org.jclouds.compute.callables.AuthorizeRSAPublicKey@5f8f127c=java.util.concurrent.ExecutionException:
org.jclouds.ssh.SshException: ec2-user@184.73.14.241:22: Error
connecting to sftp.}
at org.jclouds.compute.util.ComputeUtils.runCallablesUsingSshClient(ComputeUtils.java:247)
at org.jclouds.compute.util.ComputeUtils.runTasksUsingSshClient(ComputeUtils.java:214)
at org.jclouds.compute.util.ComputeUtils.runCallablesOnNode(ComputeUtils.java:203)
at org.jclouds.compute.util.ComputeUtils.runOptionsOnNode(ComputeUtils.java:151)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:116)
at org.jclouds.compute.util.ComputeUtils$1.call(ComputeUtils.java:112)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
1 error[s]
at org.jclouds.compute.internal.BaseComputeService.runNodesWithTag(BaseComputeService.java:162)
at org.apache.whirr.service.cassandra.CassandraService.launchCluster(CassandraService.java:109)
... 3 more
On Sat, Jan 29, 2011 at 9:05 PM, Lars George <la...@gmail.com> wrote:
> Hi Ashish,
>
> Looking at the names of your certificate files it seems you have created them up on AWS and downloaded them to your machine - that was how it also had to be done in the past with the previous EC2 scripts. Note though that this is different for Whirr, it requires an OpenSSH key pair as used by your ssh client. Those are stored in $HOME/.ssh/id_rsa (plus the matching id_rsa.pub for the public key). Look into those to see how their format looks like I bet the ones you are using are different.
>
> The easiest is really to use your existing SSH key or create one with
>
> $ ssh-keygen -t rsa
>
> And specify a different directory and point the Whirr .properties to it.
>
> Lars
>
> On Jan 29, 2011, at 14:13, Ashish <pa...@gmail.com> wrote:
>
>> Thanks Andrei for the wonderful explanation :)
>>
>> Unfortunately I am still getting some error's
>>
>> Here is my config file
>>
>> whirr.service-name=cassandra
>> whirr.cluster-name=mycassandracluster
>> whirr.instance-templates=3 cassandra
>> whirr.provider=ec2
>> whirr.identity=XXXXXXXXXX
>> whirr.credential=XXXXXXXXXX
>> whirr.private-key-file=pk-XXXXXXXXXXXX.pem
>> whirr.public-key-file=rsa-XXXXXXXXXXXX.pem
>>
>> Earlier I was using cygwin on windows. I ran this test on Mac today.
>> Got following error
>>
>> ssh, completed: 0/2, errors: 1, rate: 2288ms/op
>> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
>> ec2-user@50.16.22.161:22: Error putting path: .ssh/id_rsa.pub
>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:680)
>> Caused by: org.jclouds.ssh.SshException: ec2-user@50.16.22.161:22:
>> Error putting path: .ssh/id_rsa.pub
>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
>> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> ... 3 more
>> Caused by: 4: java.io.IOException: Pipe closed
>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
>> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
>> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
>> ... 7 more
>> Caused by: java.io.IOException: Pipe closed
>> at java.io.PipedInputStream.read(PipedInputStream.java:291)
>> at java.io.PipedInputStream.read(PipedInputStream.java:361)
>> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
>> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
>> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:474)
>> ... 10 more
>>
>>
>> From the logs I found this
>>
>> 2011-01-29 18:31:54,075 DEBUG [jclouds.compute] (main) >> creating
>> keyPair region(us-east-1) tag(mycassandracluster)
>> 2011-01-29 18:31:56,490 DEBUG [jclouds.compute] (main) << created
>> keyPair(jclouds#mycassandracluster#us-east-1#21)
>> 2011-01-29 18:31:56,491 DEBUG [jclouds.compute] (main) >> creating
>> securityGroup region(us-east-1)
>> name(jclouds#mycassandracluster#us-east-1)
>>
>>
>> seems like the keypair is being created irrespective of the settings
>> in config file.
>>
>> Am I missing something here?
>>
>> Appreciate all your help.
>>
>> thanks
>> ashish
>>
>> On Sat, Jan 29, 2011 at 5:39 PM, Andrei Savu <sa...@gmail.com> wrote:
>>>>
>>>> Two questions
>>>> 1. Is there a way to get the keypair created by whirr?
>>>
>>> You can specify the keypair used to access the server in the cluster
>>> configuration file. By default Whirr uses ~/.ssh/id_rsa and
>>> ~/.ssh/id_rsa.pub.
>>>
>>> This following snippet was taken from a recipe:
>>>
>>> # By default use the user system SSH keys. Override them here.
>>> # whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
>>> # whirr.public-key-file=${whirr.private-key-file}.pub
>>>
>>> You can override the keys specified in the configuration file on the
>>> command line:
>>>
>>> --private-key-file
>>> --public-key-file
>>>
>>> Whirr can only work with SSH RSA keypairs for now.
>>>
>>> Note: If you are creating a Hadoop or HBase cluster, Whirr creates
>>> some utility scripts that you can use to setup a local SOCKS proxy to
>>> the cluster. Check the configuration guide for more info
>>> http://incubator.apache.org/whirr/quick-start-guide.html#Run_a_proxy
>>>
>>> Also, keep in mind that Whirr does not create a user on the servers
>>> with the same name as your local user.
>>>
>>> Login to EC2 by using: $ ssh -i ssh_rsa_private_key
>>> ec2-user@public-ec2-server-ip
>>>
>>> Login to Rackspace Cloud by using: $ ssh -i ssh_rsa_private_key
>>> root@public-rackspace-cloud-ip
>>>
>>> I hope this clarifies things a bit for you. Let me know if you have
>>> more questions.
>>>
>>>>
>>>> BTW using whirr for Hadoop has been a pleasure, it rocks :)
>>>
>>> Glad you like it!
>>>
>>>>
>>>> thanks
>>>> ashish
>>>>
>>>> On Sat, Jan 29, 2011 at 12:46 AM, Tom White <to...@gmail.com> wrote:
>>>>> On Fri, Jan 28, 2011 at 12:53 AM, Ashish <pa...@gmail.com> wrote:
>>>>>> Folks,
>>>>>>
>>>>>> I followed the instructions from
>>>>>> http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
>>>>>>
>>>>>> Using whirr-0.2.0-incubating stable release
>>>>>
>>>>> I suggest trying with 0.3.0 (out soon, or available from svn now, as
>>>>> the blog outlines) since the Cassandra code has changed quite a bit
>>>>> since 0.2.0.
>>>>>
>>>>>>
>>>>>> Instances are launched, but at the end it it displays an error while
>>>>>> connecting it.
>>>>>>
>>>>>> Following exception is printed -----------------
>>>>>>
>>>>>> Authorizing firewall
>>>>>> Running configuration script
>>>>>> Exception in thread "main" java.io.IOException:
>>>>>> org.jclouds.compute.RunScriptOnNodesException: error runScript on
>>>>>> filter
>>>>>> ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
>>>>>> Execution failures:
>>>>>>
>>>>>> 0 error[s]
>>>>>> Node failures:
>>>>>>
>>>>>> 1) SshException on node us-east-1/i-17f3497b:
>>>>>> org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
>>>>>> connecting to session.
>>>>>> at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
>>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
>>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
>>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
>>>>>> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>>>>>> at java.util.concurrent.FutureTask.run(Unknown Source)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>>>>> Source)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>>>>> at java.lang.Thread.run(Unknown Source)
>>>>>> Caused by: com.jcraft.jsch.JSchException: Auth fail
>>>>>> at com.jcraft.jsch.Session.connect(Session.java:452)
>>>>>> at com.jcraft.jsch.Session.connect(Session.java:150)
>>>>>> at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
>>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
>>>>>> ... 7 more
>>>>>>
>>>>>>
>>>>>> Hadoop cluster runs fine with the whirr release.
>>>>>>
>>>>>> My configuration file is plan simple, picked from the blog and the
>>>>>> real value replaced.
>>>>>>
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Error while running cassandra using whirr
Posted by Lars George <la...@gmail.com>.
Hi Ashish,
Looking at the names of your certificate files it seems you have created them up on AWS and downloaded them to your machine - that was how it also had to be done in the past with the previous EC2 scripts. Note though that this is different for Whirr, it requires an OpenSSH key pair as used by your ssh client. Those are stored in $HOME/.ssh/id_rsa (plus the matching id_rsa.pub for the public key). Look into those to see how their format looks like I bet the ones you are using are different.
The easiest is really to use your existing SSH key or create one with
$ ssh-keygen -t rsa
And specify a different directory and point the Whirr .properties to it.
Lars
On Jan 29, 2011, at 14:13, Ashish <pa...@gmail.com> wrote:
> Thanks Andrei for the wonderful explanation :)
>
> Unfortunately I am still getting some error's
>
> Here is my config file
>
> whirr.service-name=cassandra
> whirr.cluster-name=mycassandracluster
> whirr.instance-templates=3 cassandra
> whirr.provider=ec2
> whirr.identity=XXXXXXXXXX
> whirr.credential=XXXXXXXXXX
> whirr.private-key-file=pk-XXXXXXXXXXXX.pem
> whirr.public-key-file=rsa-XXXXXXXXXXXX.pem
>
> Earlier I was using cygwin on windows. I ran this test on Mac today.
> Got following error
>
> ssh, completed: 0/2, errors: 1, rate: 2288ms/op
> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
> ec2-user@50.16.22.161:22: Error putting path: .ssh/id_rsa.pub
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: org.jclouds.ssh.SshException: ec2-user@50.16.22.161:22:
> Error putting path: .ssh/id_rsa.pub
> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> Caused by: 4: java.io.IOException: Pipe closed
> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
> ... 7 more
> Caused by: java.io.IOException: Pipe closed
> at java.io.PipedInputStream.read(PipedInputStream.java:291)
> at java.io.PipedInputStream.read(PipedInputStream.java:361)
> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:474)
> ... 10 more
>
>
> From the logs I found this
>
> 2011-01-29 18:31:54,075 DEBUG [jclouds.compute] (main) >> creating
> keyPair region(us-east-1) tag(mycassandracluster)
> 2011-01-29 18:31:56,490 DEBUG [jclouds.compute] (main) << created
> keyPair(jclouds#mycassandracluster#us-east-1#21)
> 2011-01-29 18:31:56,491 DEBUG [jclouds.compute] (main) >> creating
> securityGroup region(us-east-1)
> name(jclouds#mycassandracluster#us-east-1)
>
>
> seems like the keypair is being created irrespective of the settings
> in config file.
>
> Am I missing something here?
>
> Appreciate all your help.
>
> thanks
> ashish
>
> On Sat, Jan 29, 2011 at 5:39 PM, Andrei Savu <sa...@gmail.com> wrote:
>>>
>>> Two questions
>>> 1. Is there a way to get the keypair created by whirr?
>>
>> You can specify the keypair used to access the server in the cluster
>> configuration file. By default Whirr uses ~/.ssh/id_rsa and
>> ~/.ssh/id_rsa.pub.
>>
>> This following snippet was taken from a recipe:
>>
>> # By default use the user system SSH keys. Override them here.
>> # whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
>> # whirr.public-key-file=${whirr.private-key-file}.pub
>>
>> You can override the keys specified in the configuration file on the
>> command line:
>>
>> --private-key-file
>> --public-key-file
>>
>> Whirr can only work with SSH RSA keypairs for now.
>>
>> Note: If you are creating a Hadoop or HBase cluster, Whirr creates
>> some utility scripts that you can use to setup a local SOCKS proxy to
>> the cluster. Check the configuration guide for more info
>> http://incubator.apache.org/whirr/quick-start-guide.html#Run_a_proxy
>>
>> Also, keep in mind that Whirr does not create a user on the servers
>> with the same name as your local user.
>>
>> Login to EC2 by using: $ ssh -i ssh_rsa_private_key
>> ec2-user@public-ec2-server-ip
>>
>> Login to Rackspace Cloud by using: $ ssh -i ssh_rsa_private_key
>> root@public-rackspace-cloud-ip
>>
>> I hope this clarifies things a bit for you. Let me know if you have
>> more questions.
>>
>>>
>>> BTW using whirr for Hadoop has been a pleasure, it rocks :)
>>
>> Glad you like it!
>>
>>>
>>> thanks
>>> ashish
>>>
>>> On Sat, Jan 29, 2011 at 12:46 AM, Tom White <to...@gmail.com> wrote:
>>>> On Fri, Jan 28, 2011 at 12:53 AM, Ashish <pa...@gmail.com> wrote:
>>>>> Folks,
>>>>>
>>>>> I followed the instructions from
>>>>> http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
>>>>>
>>>>> Using whirr-0.2.0-incubating stable release
>>>>
>>>> I suggest trying with 0.3.0 (out soon, or available from svn now, as
>>>> the blog outlines) since the Cassandra code has changed quite a bit
>>>> since 0.2.0.
>>>>
>>>>>
>>>>> Instances are launched, but at the end it it displays an error while
>>>>> connecting it.
>>>>>
>>>>> Following exception is printed -----------------
>>>>>
>>>>> Authorizing firewall
>>>>> Running configuration script
>>>>> Exception in thread "main" java.io.IOException:
>>>>> org.jclouds.compute.RunScriptOnNodesException: error runScript on
>>>>> filter
>>>>> ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
>>>>> Execution failures:
>>>>>
>>>>> 0 error[s]
>>>>> Node failures:
>>>>>
>>>>> 1) SshException on node us-east-1/i-17f3497b:
>>>>> org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
>>>>> connecting to session.
>>>>> at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
>>>>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
>>>>> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>>>>> at java.util.concurrent.FutureTask.run(Unknown Source)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>>>> Source)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>>>> at java.lang.Thread.run(Unknown Source)
>>>>> Caused by: com.jcraft.jsch.JSchException: Auth fail
>>>>> at com.jcraft.jsch.Session.connect(Session.java:452)
>>>>> at com.jcraft.jsch.Session.connect(Session.java:150)
>>>>> at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
>>>>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
>>>>> ... 7 more
>>>>>
>>>>>
>>>>> Hadoop cluster runs fine with the whirr release.
>>>>>
>>>>> My configuration file is plan simple, picked from the blog and the
>>>>> real value replaced.
>>>>>
Re: Error while running cassandra using whirr
Posted by Andrei Savu <sa...@gmail.com>.
Are you using 0.3.0 from svn or 0.2.0?
Whirr 0.3.0 does strict SSH RSA keypair validation and that should be
helpful for you.
I suspect there is something wrong with the keys. How are you
generating them? Using ssh-keygen?
On Sat, Jan 29, 2011 at 3:13 PM, Ashish <pa...@gmail.com> wrote:
> Thanks Andrei for the wonderful explanation :)
>
> Unfortunately I am still getting some error's
>
> Here is my config file
>
> whirr.service-name=cassandra
> whirr.cluster-name=mycassandracluster
> whirr.instance-templates=3 cassandra
> whirr.provider=ec2
> whirr.identity=XXXXXXXXXX
> whirr.credential=XXXXXXXXXX
> whirr.private-key-file=pk-XXXXXXXXXXXX.pem
> whirr.public-key-file=rsa-XXXXXXXXXXXX.pem
>
> Earlier I was using cygwin on windows. I ran this test on Mac today.
> Got following error
>
> ssh, completed: 0/2, errors: 1, rate: 2288ms/op
> java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
> ec2-user@50.16.22.161:22: Error putting path: .ssh/id_rsa.pub
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: org.jclouds.ssh.SshException: ec2-user@50.16.22.161:22:
> Error putting path: .ssh/id_rsa.pub
> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
> at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> Caused by: 4: java.io.IOException: Pipe closed
> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
> at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
> at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
> ... 7 more
> Caused by: java.io.IOException: Pipe closed
> at java.io.PipedInputStream.read(PipedInputStream.java:291)
> at java.io.PipedInputStream.read(PipedInputStream.java:361)
> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
> at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:474)
> ... 10 more
>
>
> From the logs I found this
>
> 2011-01-29 18:31:54,075 DEBUG [jclouds.compute] (main) >> creating
> keyPair region(us-east-1) tag(mycassandracluster)
> 2011-01-29 18:31:56,490 DEBUG [jclouds.compute] (main) << created
> keyPair(jclouds#mycassandracluster#us-east-1#21)
> 2011-01-29 18:31:56,491 DEBUG [jclouds.compute] (main) >> creating
> securityGroup region(us-east-1)
> name(jclouds#mycassandracluster#us-east-1)
>
>
> seems like the keypair is being created irrespective of the settings
> in config file.
>
> Am I missing something here?
>
> Appreciate all your help.
>
> thanks
> ashish
>
> On Sat, Jan 29, 2011 at 5:39 PM, Andrei Savu <sa...@gmail.com> wrote:
>>>
>>> Two questions
>>> 1. Is there a way to get the keypair created by whirr?
>>
>> You can specify the keypair used to access the server in the cluster
>> configuration file. By default Whirr uses ~/.ssh/id_rsa and
>> ~/.ssh/id_rsa.pub.
>>
>> This following snippet was taken from a recipe:
>>
>> # By default use the user system SSH keys. Override them here.
>> # whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
>> # whirr.public-key-file=${whirr.private-key-file}.pub
>>
>> You can override the keys specified in the configuration file on the
>> command line:
>>
>> --private-key-file
>> --public-key-file
>>
>> Whirr can only work with SSH RSA keypairs for now.
>>
>> Note: If you are creating a Hadoop or HBase cluster, Whirr creates
>> some utility scripts that you can use to setup a local SOCKS proxy to
>> the cluster. Check the configuration guide for more info
>> http://incubator.apache.org/whirr/quick-start-guide.html#Run_a_proxy
>>
>> Also, keep in mind that Whirr does not create a user on the servers
>> with the same name as your local user.
>>
>> Login to EC2 by using: $ ssh -i ssh_rsa_private_key
>> ec2-user@public-ec2-server-ip
>>
>> Login to Rackspace Cloud by using: $ ssh -i ssh_rsa_private_key
>> root@public-rackspace-cloud-ip
>>
>> I hope this clarifies things a bit for you. Let me know if you have
>> more questions.
>>
>>>
>>> BTW using whirr for Hadoop has been a pleasure, it rocks :)
>>
>> Glad you like it!
>>
>>>
>>> thanks
>>> ashish
>>>
>>> On Sat, Jan 29, 2011 at 12:46 AM, Tom White <to...@gmail.com> wrote:
>>> > On Fri, Jan 28, 2011 at 12:53 AM, Ashish <pa...@gmail.com> wrote:
>>> >> Folks,
>>> >>
>>> >> I followed the instructions from
>>> >> http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
>>> >>
>>> >> Using whirr-0.2.0-incubating stable release
>>> >
>>> > I suggest trying with 0.3.0 (out soon, or available from svn now, as
>>> > the blog outlines) since the Cassandra code has changed quite a bit
>>> > since 0.2.0.
>>> >
>>> >>
>>> >> Instances are launched, but at the end it it displays an error while
>>> >> connecting it.
>>> >>
>>> >> Following exception is printed -----------------
>>> >>
>>> >> Authorizing firewall
>>> >> Running configuration script
>>> >> Exception in thread "main" java.io.IOException:
>>> >> org.jclouds.compute.RunScriptOnNodesException: error runScript on
>>> >> filter
>>> >> ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
>>> >> Execution failures:
>>> >>
>>> >> 0 error[s]
>>> >> Node failures:
>>> >>
>>> >> 1) SshException on node us-east-1/i-17f3497b:
>>> >> org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
>>> >> connecting to session.
>>> >> at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
>>> >> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
>>> >> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
>>> >> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
>>> >> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>>> >> at java.util.concurrent.FutureTask.run(Unknown Source)
>>> >> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>> >> Source)
>>> >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>> >> at java.lang.Thread.run(Unknown Source)
>>> >> Caused by: com.jcraft.jsch.JSchException: Auth fail
>>> >> at com.jcraft.jsch.Session.connect(Session.java:452)
>>> >> at com.jcraft.jsch.Session.connect(Session.java:150)
>>> >> at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
>>> >> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
>>> >> ... 7 more
>>> >>
>>> >>
>>> >> Hadoop cluster runs fine with the whirr release.
>>> >>
>>> >> My configuration file is plan simple, picked from the blog and the
>>> >> real value replaced.
>>> >>
>
--
Andrei Savu -- andreisavu.ro
Re: Error while running cassandra using whirr
Posted by Ashish <pa...@gmail.com>.
Thanks Andrei for the wonderful explanation :)
Unfortunately I am still getting some error's
Here is my config file
whirr.service-name=cassandra
whirr.cluster-name=mycassandracluster
whirr.instance-templates=3 cassandra
whirr.provider=ec2
whirr.identity=XXXXXXXXXX
whirr.credential=XXXXXXXXXX
whirr.private-key-file=pk-XXXXXXXXXXXX.pem
whirr.public-key-file=rsa-XXXXXXXXXXXX.pem
Earlier I was using cygwin on windows. I ran this test on Mac today.
Got following error
ssh, completed: 0/2, errors: 1, rate: 2288ms/op
java.util.concurrent.ExecutionException: org.jclouds.ssh.SshException:
ec2-user@50.16.22.161:22: Error putting path: .ssh/id_rsa.pub
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:121)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.jclouds.ssh.SshException: ec2-user@50.16.22.161:22:
Error putting path: .ssh/id_rsa.pub
at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:167)
at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:52)
at org.jclouds.compute.callables.AuthorizeRSAPublicKey.call(AuthorizeRSAPublicKey.java:37)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: 4: java.io.IOException: Pipe closed
at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:578)
at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:439)
at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:406)
at org.jclouds.ssh.jsch.JschSshClient.put(JschSshClient.java:165)
... 7 more
Caused by: java.io.IOException: Pipe closed
at java.io.PipedInputStream.read(PipedInputStream.java:291)
at java.io.PipedInputStream.read(PipedInputStream.java:361)
at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2325)
at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2351)
at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:474)
... 10 more
>From the logs I found this
2011-01-29 18:31:54,075 DEBUG [jclouds.compute] (main) >> creating
keyPair region(us-east-1) tag(mycassandracluster)
2011-01-29 18:31:56,490 DEBUG [jclouds.compute] (main) << created
keyPair(jclouds#mycassandracluster#us-east-1#21)
2011-01-29 18:31:56,491 DEBUG [jclouds.compute] (main) >> creating
securityGroup region(us-east-1)
name(jclouds#mycassandracluster#us-east-1)
seems like the keypair is being created irrespective of the settings
in config file.
Am I missing something here?
Appreciate all your help.
thanks
ashish
On Sat, Jan 29, 2011 at 5:39 PM, Andrei Savu <sa...@gmail.com> wrote:
>>
>> Two questions
>> 1. Is there a way to get the keypair created by whirr?
>
> You can specify the keypair used to access the server in the cluster
> configuration file. By default Whirr uses ~/.ssh/id_rsa and
> ~/.ssh/id_rsa.pub.
>
> This following snippet was taken from a recipe:
>
> # By default use the user system SSH keys. Override them here.
> # whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
> # whirr.public-key-file=${whirr.private-key-file}.pub
>
> You can override the keys specified in the configuration file on the
> command line:
>
> --private-key-file
> --public-key-file
>
> Whirr can only work with SSH RSA keypairs for now.
>
> Note: If you are creating a Hadoop or HBase cluster, Whirr creates
> some utility scripts that you can use to setup a local SOCKS proxy to
> the cluster. Check the configuration guide for more info
> http://incubator.apache.org/whirr/quick-start-guide.html#Run_a_proxy
>
> Also, keep in mind that Whirr does not create a user on the servers
> with the same name as your local user.
>
> Login to EC2 by using: $ ssh -i ssh_rsa_private_key
> ec2-user@public-ec2-server-ip
>
> Login to Rackspace Cloud by using: $ ssh -i ssh_rsa_private_key
> root@public-rackspace-cloud-ip
>
> I hope this clarifies things a bit for you. Let me know if you have
> more questions.
>
>>
>> BTW using whirr for Hadoop has been a pleasure, it rocks :)
>
> Glad you like it!
>
>>
>> thanks
>> ashish
>>
>> On Sat, Jan 29, 2011 at 12:46 AM, Tom White <to...@gmail.com> wrote:
>> > On Fri, Jan 28, 2011 at 12:53 AM, Ashish <pa...@gmail.com> wrote:
>> >> Folks,
>> >>
>> >> I followed the instructions from
>> >> http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
>> >>
>> >> Using whirr-0.2.0-incubating stable release
>> >
>> > I suggest trying with 0.3.0 (out soon, or available from svn now, as
>> > the blog outlines) since the Cassandra code has changed quite a bit
>> > since 0.2.0.
>> >
>> >>
>> >> Instances are launched, but at the end it it displays an error while
>> >> connecting it.
>> >>
>> >> Following exception is printed -----------------
>> >>
>> >> Authorizing firewall
>> >> Running configuration script
>> >> Exception in thread "main" java.io.IOException:
>> >> org.jclouds.compute.RunScriptOnNodesException: error runScript on
>> >> filter
>> >> ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
>> >> Execution failures:
>> >>
>> >> 0 error[s]
>> >> Node failures:
>> >>
>> >> 1) SshException on node us-east-1/i-17f3497b:
>> >> org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
>> >> connecting to session.
>> >> at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
>> >> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
>> >> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
>> >> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
>> >> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>> >> at java.util.concurrent.FutureTask.run(Unknown Source)
>> >> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>> >> Source)
>> >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>> >> at java.lang.Thread.run(Unknown Source)
>> >> Caused by: com.jcraft.jsch.JSchException: Auth fail
>> >> at com.jcraft.jsch.Session.connect(Session.java:452)
>> >> at com.jcraft.jsch.Session.connect(Session.java:150)
>> >> at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
>> >> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
>> >> ... 7 more
>> >>
>> >>
>> >> Hadoop cluster runs fine with the whirr release.
>> >>
>> >> My configuration file is plan simple, picked from the blog and the
>> >> real value replaced.
>> >>
Re: Error while running cassandra using whirr
Posted by Andrei Savu <sa...@gmail.com>.
>
> Two questions
> 1. Is there a way to get the keypair created by whirr?
You can specify the keypair used to access the server in the cluster
configuration file. By default Whirr uses ~/.ssh/id_rsa and
~/.ssh/id_rsa.pub.
This following snippet was taken from a recipe:
# By default use the user system SSH keys. Override them here.
# whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
# whirr.public-key-file=${whirr.private-key-file}.pub
You can override the keys specified in the configuration file on the
command line:
--private-key-file
--public-key-file
Whirr can only work with SSH RSA keypairs for now.
Note: If you are creating a Hadoop or HBase cluster, Whirr creates
some utility scripts that you can use to setup a local SOCKS proxy to
the cluster. Check the configuration guide for more info
http://incubator.apache.org/whirr/quick-start-guide.html#Run_a_proxy
Also, keep in mind that Whirr does not create a user on the servers
with the same name as your local user.
Login to EC2 by using: $ ssh -i ssh_rsa_private_key
ec2-user@public-ec2-server-ip
Login to Rackspace Cloud by using: $ ssh -i ssh_rsa_private_key
root@public-rackspace-cloud-ip
I hope this clarifies things a bit for you. Let me know if you have
more questions.
>
> BTW using whirr for Hadoop has been a pleasure, it rocks :)
Glad you like it!
>
> thanks
> ashish
>
> On Sat, Jan 29, 2011 at 12:46 AM, Tom White <to...@gmail.com> wrote:
> > On Fri, Jan 28, 2011 at 12:53 AM, Ashish <pa...@gmail.com> wrote:
> >> Folks,
> >>
> >> I followed the instructions from
> >> http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
> >>
> >> Using whirr-0.2.0-incubating stable release
> >
> > I suggest trying with 0.3.0 (out soon, or available from svn now, as
> > the blog outlines) since the Cassandra code has changed quite a bit
> > since 0.2.0.
> >
> >>
> >> Instances are launched, but at the end it it displays an error while
> >> connecting it.
> >>
> >> Following exception is printed -----------------
> >>
> >> Authorizing firewall
> >> Running configuration script
> >> Exception in thread "main" java.io.IOException:
> >> org.jclouds.compute.RunScriptOnNodesException: error runScript on
> >> filter
> >> ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
> >> Execution failures:
> >>
> >> 0 error[s]
> >> Node failures:
> >>
> >> 1) SshException on node us-east-1/i-17f3497b:
> >> org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
> >> connecting to session.
> >> at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
> >> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
> >> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
> >> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
> >> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
> >> at java.util.concurrent.FutureTask.run(Unknown Source)
> >> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> >> Source)
> >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> >> at java.lang.Thread.run(Unknown Source)
> >> Caused by: com.jcraft.jsch.JSchException: Auth fail
> >> at com.jcraft.jsch.Session.connect(Session.java:452)
> >> at com.jcraft.jsch.Session.connect(Session.java:150)
> >> at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
> >> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
> >> ... 7 more
> >>
> >>
> >> Hadoop cluster runs fine with the whirr release.
> >>
> >> My configuration file is plan simple, picked from the blog and the
> >> real value replaced.
> >>
> >> Am I missing anything out here?
> >>
> >> thanks
> >> ashish
> >>
> >
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
--
Andrei Savu -- andreisavu.ro
Re: Error while running cassandra using whirr
Posted by Ashish <pa...@gmail.com>.
Sure things, will wait for the release.
There is one thing I noted, whirr creates a keypair and registers it
with instances.
I assume the same would be needed to login to the servers.
I used AWS Management console for verify this, and upon clicking
Connect to an instances it says use the
keypair (created by whirr).
Two questions
1. Is there a way to get the keypair created by whirr?
2. Is there a possibility to specify a keypair rather than creating a new one?
BTW using whirr for Hadoop has been a pleasure, it rocks :)
thanks
ashish
On Sat, Jan 29, 2011 at 12:46 AM, Tom White <to...@gmail.com> wrote:
> On Fri, Jan 28, 2011 at 12:53 AM, Ashish <pa...@gmail.com> wrote:
>> Folks,
>>
>> I followed the instructions from
>> http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
>>
>> Using whirr-0.2.0-incubating stable release
>
> I suggest trying with 0.3.0 (out soon, or available from svn now, as
> the blog outlines) since the Cassandra code has changed quite a bit
> since 0.2.0.
>
>>
>> Instances are launched, but at the end it it displays an error while
>> connecting it.
>>
>> Following exception is printed -----------------
>>
>> Authorizing firewall
>> Running configuration script
>> Exception in thread "main" java.io.IOException:
>> org.jclouds.compute.RunScriptOnNodesException: error runScript on
>> filter
>> ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
>> Execution failures:
>>
>> 0 error[s]
>> Node failures:
>>
>> 1) SshException on node us-east-1/i-17f3497b:
>> org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
>> connecting to session.
>> at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
>> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
>> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>> at java.util.concurrent.FutureTask.run(Unknown Source)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>> Source)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>> at java.lang.Thread.run(Unknown Source)
>> Caused by: com.jcraft.jsch.JSchException: Auth fail
>> at com.jcraft.jsch.Session.connect(Session.java:452)
>> at com.jcraft.jsch.Session.connect(Session.java:150)
>> at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
>> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
>> ... 7 more
>>
>>
>> Hadoop cluster runs fine with the whirr release.
>>
>> My configuration file is plan simple, picked from the blog and the
>> real value replaced.
>>
>> Am I missing anything out here?
>>
>> thanks
>> ashish
>>
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: Error while running cassandra using whirr
Posted by Tom White <to...@gmail.com>.
On Fri, Jan 28, 2011 at 12:53 AM, Ashish <pa...@gmail.com> wrote:
> Folks,
>
> I followed the instructions from
> http://www.philwhln.com/quickly-launch-a-cassandra-cluster-on-amazon-ec2
>
> Using whirr-0.2.0-incubating stable release
I suggest trying with 0.3.0 (out soon, or available from svn now, as
the blog outlines) since the Cassandra code has changed quite a bit
since 0.2.0.
>
> Instances are launched, but at the end it it displays an error while
> connecting it.
>
> Following exception is printed -----------------
>
> Authorizing firewall
> Running configuration script
> Exception in thread "main" java.io.IOException:
> org.jclouds.compute.RunScriptOnNodesException: error runScript on
> filter
> ed nodes options(RunScriptOptions [overridingCredentials=true, runAsRoot=true])
> Execution failures:
>
> 0 error[s]
> Node failures:
>
> 1) SshException on node us-east-1/i-17f3497b:
> org.jclouds.ssh.SshException: ec2-user@50.16.165.161:22: Error
> connecting to session.
> at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:250)
> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:204)
> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:375)
> at org.jclouds.compute.internal.BaseComputeService$4.call(BaseComputeService.java:364)
> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: com.jcraft.jsch.JSchException: Auth fail
> at com.jcraft.jsch.Session.connect(Session.java:452)
> at com.jcraft.jsch.Session.connect(Session.java:150)
> at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:245)
> at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:184)
> ... 7 more
>
>
> Hadoop cluster runs fine with the whirr release.
>
> My configuration file is plan simple, picked from the blog and the
> real value replaced.
>
> Am I missing anything out here?
>
> thanks
> ashish
>