You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Alton Alexander <al...@gmail.com> on 2014/04/11 21:37:08 UTC
0.9 wont start cluster on ec2, SSH connection refused?
I run the follwoing command and it correctly starts one head and one
master but then it fails because it can't log onto the head with the
ssh key. The wierd thing is that I can log onto the head with that
same public key. (ssh -i myamazonkey.pem
root@ec2-54-86-3-208.compute-1.amazonaws.com)
Thanks in advance!
$ spark-0.9.1-bin-hadoop2/ec2/spark-ec2 -k myamazonkey -i
~/myamazonkey.pem -s 1 launch spark-test-cluster
Setting up security groups...
Searching for existing cluster spark-test-cluster...
Spark AMI: ami-5bb18832
Launching instances...
Launched 1 slaves in us-east-1c, regid = r-8b73b4a8
Launched master in us-east-1c, regid = r-ea76b1c9
Waiting for instances to start up...
Waiting 120 more seconds...
Generating cluster's SSH key on master...
ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
Connection refused
Error executing remote command, retrying after 30 seconds: Command
'['ssh', '-o', 'StrictHostKeyChecking=no', '-i',
'/home/ec2-user/myamazonkey.pem', '-t', '-t',
u'root@ec2-54-86-3-208.compute-1.amazonaws.com', "\n [ -f
~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f
~/.ssh/id_rsa &&\n cat ~/.ssh/id_rsa.pub >>
~/.ssh/authorized_keys)\n "]' returned non-zero exit status 255
ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
Connection refused
Error executing remote command, retrying after 30 seconds: Command
'['ssh', '-o', 'StrictHostKeyChecking=no', '-i',
'/home/ec2-user/myamazonkey.pem', '-t', '-t',
u'root@ec2-54-86-3-208.compute-1.amazonaws.com', "\n [ -f
~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f
~/.ssh/id_rsa &&\n cat ~/.ssh/id_rsa.pub >>
~/.ssh/authorized_keys)\n "]' returned non-zero exit status 255
ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
Connection refused
Error executing remote command, retrying after 30 seconds: Command
'['ssh', '-o', 'StrictHostKeyChecking=no', '-i',
'/home/ec2-user/myamazonkey.pem', '-t', '-t',
u'root@ec2-54-86-3-208.compute-1.amazonaws.com', "\n [ -f
~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f
~/.ssh/id_rsa &&\n cat ~/.ssh/id_rsa.pub >>
~/.ssh/authorized_keys)\n "]' returned non-zero exit status 255
ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
Connection refused
Error:
Failed to SSH to remote host ec2-54-86-3-208.compute-1.amazonaws.com.
Please check that you have provided the correct --identity-file and
--key-pair parameters and try again.
Re: 0.9 wont start cluster on ec2, SSH connection refused?
Posted by wxhsdp <wx...@gmail.com>.
Hi, mayur
i've met the same problem. the instances are on, i can see them from ec2
console, and connect to them
wxhsdp@ubuntu:~/spark/spark/tags/v1.0.0-rc3/ec2$ ssh -i wxhsdp-us-east.pem
root@54.86.181.108
The authenticity of host '54.86.181.108 (54.86.181.108)' can't be
established.
ECDSA key fingerprint is
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '54.86.181.108' (ECDSA) to the list of known
hosts.
Last login: Sun Feb 2 23:13:21 2014 from
c-50-131-222-227.hsd1.ca.comcast.net
__| __|_ )
_| ( / Amazon Linux AMI
___|\___|___|
https://aws.amazon.com/amazon-linux-ami/2013.03-release-notes/
There are 42 security update(s) out of 257 total update(s) available
Run "sudo yum update" to apply all updates.
Amazon Linux version 2014.03 is available.
but there's /root/ephemeral-hdfs and spark directory as said in the
ec2-scripts.html
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/0-9-wont-start-cluster-on-ec2-SSH-connection-refused-tp4146p5498.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: 0.9 wont start cluster on ec2, SSH connection refused?
Posted by Alton Alexander <al...@gmail.com>.
No not anymore but it was at the time. Thanks but I also just found a
thread from two days ago discussing the root and es2-user workaround.
For now I'll just go back to using the AMI provided.
Thanks!
On Fri, Apr 11, 2014 at 1:39 PM, Mayur Rustagi <ma...@gmail.com> wrote:
> is the machine booted up & reachable?
>
> Mayur Rustagi
> Ph: +1 (760) 203 3257
> http://www.sigmoidanalytics.com
> @mayur_rustagi
>
>
>
> On Fri, Apr 11, 2014 at 12:37 PM, Alton Alexander <al...@gmail.com>
> wrote:
>>
>> I run the follwoing command and it correctly starts one head and one
>> master but then it fails because it can't log onto the head with the
>> ssh key. The wierd thing is that I can log onto the head with that
>> same public key. (ssh -i myamazonkey.pem
>> root@ec2-54-86-3-208.compute-1.amazonaws.com)
>>
>> Thanks in advance!
>>
>> $ spark-0.9.1-bin-hadoop2/ec2/spark-ec2 -k myamazonkey -i
>> ~/myamazonkey.pem -s 1 launch spark-test-cluster
>>
>> Setting up security groups...
>> Searching for existing cluster spark-test-cluster...
>> Spark AMI: ami-5bb18832
>> Launching instances...
>> Launched 1 slaves in us-east-1c, regid = r-8b73b4a8
>> Launched master in us-east-1c, regid = r-ea76b1c9
>> Waiting for instances to start up...
>> Waiting 120 more seconds...
>> Generating cluster's SSH key on master...
>> ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
>> Connection refused
>> Error executing remote command, retrying after 30 seconds: Command
>> '['ssh', '-o', 'StrictHostKeyChecking=no', '-i',
>> '/home/ec2-user/myamazonkey.pem', '-t', '-t',
>> u'root@ec2-54-86-3-208.compute-1.amazonaws.com', "\n [ -f
>> ~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f
>> ~/.ssh/id_rsa &&\n cat ~/.ssh/id_rsa.pub >>
>> ~/.ssh/authorized_keys)\n "]' returned non-zero exit status 255
>> ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
>> Connection refused
>> Error executing remote command, retrying after 30 seconds: Command
>> '['ssh', '-o', 'StrictHostKeyChecking=no', '-i',
>> '/home/ec2-user/myamazonkey.pem', '-t', '-t',
>> u'root@ec2-54-86-3-208.compute-1.amazonaws.com', "\n [ -f
>> ~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f
>> ~/.ssh/id_rsa &&\n cat ~/.ssh/id_rsa.pub >>
>> ~/.ssh/authorized_keys)\n "]' returned non-zero exit status 255
>> ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
>> Connection refused
>> Error executing remote command, retrying after 30 seconds: Command
>> '['ssh', '-o', 'StrictHostKeyChecking=no', '-i',
>> '/home/ec2-user/myamazonkey.pem', '-t', '-t',
>> u'root@ec2-54-86-3-208.compute-1.amazonaws.com', "\n [ -f
>> ~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f
>> ~/.ssh/id_rsa &&\n cat ~/.ssh/id_rsa.pub >>
>> ~/.ssh/authorized_keys)\n "]' returned non-zero exit status 255
>> ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
>> Connection refused
>>
>> Error:
>> Failed to SSH to remote host ec2-54-86-3-208.compute-1.amazonaws.com.
>> Please check that you have provided the correct --identity-file and
>> --key-pair parameters and try again.
>
>
Re: 0.9 wont start cluster on ec2, SSH connection refused?
Posted by Mayur Rustagi <ma...@gmail.com>.
is the machine booted up & reachable?
Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi <https://twitter.com/mayur_rustagi>
On Fri, Apr 11, 2014 at 12:37 PM, Alton Alexander
<al...@gmail.com>wrote:
> I run the follwoing command and it correctly starts one head and one
> master but then it fails because it can't log onto the head with the
> ssh key. The wierd thing is that I can log onto the head with that
> same public key. (ssh -i myamazonkey.pem
> root@ec2-54-86-3-208.compute-1.amazonaws.com)
>
> Thanks in advance!
>
> $ spark-0.9.1-bin-hadoop2/ec2/spark-ec2 -k myamazonkey -i
> ~/myamazonkey.pem -s 1 launch spark-test-cluster
>
> Setting up security groups...
> Searching for existing cluster spark-test-cluster...
> Spark AMI: ami-5bb18832
> Launching instances...
> Launched 1 slaves in us-east-1c, regid = r-8b73b4a8
> Launched master in us-east-1c, regid = r-ea76b1c9
> Waiting for instances to start up...
> Waiting 120 more seconds...
> Generating cluster's SSH key on master...
> ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
> Connection refused
> Error executing remote command, retrying after 30 seconds: Command
> '['ssh', '-o', 'StrictHostKeyChecking=no', '-i',
> '/home/ec2-user/myamazonkey.pem', '-t', '-t',
> u'root@ec2-54-86-3-208.compute-1.amazonaws.com', "\n [ -f
> ~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f
> ~/.ssh/id_rsa &&\n cat ~/.ssh/id_rsa.pub >>
> ~/.ssh/authorized_keys)\n "]' returned non-zero exit status 255
> ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
> Connection refused
> Error executing remote command, retrying after 30 seconds: Command
> '['ssh', '-o', 'StrictHostKeyChecking=no', '-i',
> '/home/ec2-user/myamazonkey.pem', '-t', '-t',
> u'root@ec2-54-86-3-208.compute-1.amazonaws.com', "\n [ -f
> ~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f
> ~/.ssh/id_rsa &&\n cat ~/.ssh/id_rsa.pub >>
> ~/.ssh/authorized_keys)\n "]' returned non-zero exit status 255
> ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
> Connection refused
> Error executing remote command, retrying after 30 seconds: Command
> '['ssh', '-o', 'StrictHostKeyChecking=no', '-i',
> '/home/ec2-user/myamazonkey.pem', '-t', '-t',
> u'root@ec2-54-86-3-208.compute-1.amazonaws.com', "\n [ -f
> ~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f
> ~/.ssh/id_rsa &&\n cat ~/.ssh/id_rsa.pub >>
> ~/.ssh/authorized_keys)\n "]' returned non-zero exit status 255
> ssh: connect to host ec2-54-86-3-208.compute-1.amazonaws.com port 22:
> Connection refused
>
> Error:
> Failed to SSH to remote host ec2-54-86-3-208.compute-1.amazonaws.com.
> Please check that you have provided the correct --identity-file and
> --key-pair parameters and try again.
>