You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Bogdan Tanasa <ta...@gmail.com> on 2021/03/22 16:53:51 UTC
connecting to an Apache Spark on AWS and port 22
Dear all,
after i am setting up an Apache Spark cluster on AWS, when i am trying to
connect to it, the message i receive is shown below.
Any suggestions would be appreciated please. Thanks a lot !
ssh -i week7.pem hadoop@ec2-54-236-38-60.compute-1.amazonaws.com
ssh: connect to host ec2-54-236-38-60.compute-1.amazonaws.com port 22:
Connection timed out ...
[image: Screenshot from 2021-03-22 09-41-11.png]
Re: connecting to an Apache Spark on AWS and port 22
Posted by Gourav Sengupta <go...@gmail.com>.
Hi,
Just as always, before I immediately jump into answering, I try to
understand the question first. With that approach it would be great if you
could kindly answer the following questions:
1. any idea why are you posting a public IP address in a public forum?
2. why are you trying to use 5.29? that is a version of SPARK that was used
almost one year back
3. why are you using m5.xlarge instance type? You are actually ending up
paying more compared the amount of data processing you can do based on CPU
processing power
4. what about those roles, have you tried to migrate to the new roles as
those roles are about to be deprecated
5. why are you starting Zeppelin, Ganglia and all other applications? Do
you need them? For an instance of type m5.xlarge that may just be an
overkill
6. have you tried to see the option "On cluster user Interfaces", it
clearly reads "Not Enabled"
7. have you clicked on the link "connect to the master node using SSH"?
8. if you are experimenting and using EMR then why not use spot instances?
9. Have you tried using EMR notebooks? They are great in terms of working
on interactive notebooks and later scheduling them.
10. Have you tried using Sagemaker notebooks in case you are developing ML/
AI based applications, they are a great place to look into for developing
end to end data applications
11. Have you tried using Glue, though AWS advertises it quite highly it is
pretty much useless when it comes to interactive data exploratory work in
terms of pricing.
The list of questions goes on, but I think I will stop here, there are
still several other places you can do things better. Best of luck.
Regards,
Gourav Sengupta
On Mon, Mar 22, 2021 at 5:02 PM Bogdan Tanasa <ta...@gmail.com> wrote:
> Dear all,
>
> after i am setting up an Apache Spark cluster on AWS, when i am trying to
> connect to it, the message i receive is shown below.
>
> Any suggestions would be appreciated please. Thanks a lot !
>
> ssh -i week7.pem hadoop@ec2-54-236-38-60.compute-1.amazonaws.com
> ssh: connect to host ec2-54-236-38-60.compute-1.amazonaws.com port 22:
> Connection timed out ...
>
> [image: Screenshot from 2021-03-22 09-41-11.png]
>
>
>
Re: connecting to an Apache Spark on AWS and port 22
Posted by hania * <ha...@gmail.com>.
Hello Bogdan,
Is your cluster deployed in VPC? You could check the configuration of your
security group, if it is allowing tcp connection to port 22:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/security-group-rules-reference.html#sg-rules-local-access
greetings,
Hania
W dniu pon., 22.03.2021 o 18:33 Mich Talebzadeh <mi...@gmail.com>
napisał(a):
> Hi,
>
> Are you connecting from on-premise via ssh to Cloud.
>
> try
>
> ssh -v <USERNAME>@<IPAddress>
>
> To see the cause of the issue
>
> HTH
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Mon, 22 Mar 2021 at 17:08, Bogdan Tanasa <ta...@gmail.com> wrote:
>
>> Dear all,
>>
>> after i am setting up an Apache Spark cluster on AWS, when i am trying to
>> connect to it, the message i receive is shown below.
>>
>> Any suggestions would be appreciated please. Thanks a lot !
>>
>> ssh -i week7.pem hadoop@ec2-54-236-38-60.compute-1.amazonaws.com
>> ssh: connect to host ec2-54-236-38-60.compute-1.amazonaws.com port 22:
>> Connection timed out ...
>>
>> [image: Screenshot from 2021-03-22 09-41-11.png]
>>
>>
>>
Re: connecting to an Apache Spark on AWS and port 22
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi,
Are you connecting from on-premise via ssh to Cloud.
try
ssh -v <USERNAME>@<IPAddress>
To see the cause of the issue
HTH
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Mon, 22 Mar 2021 at 17:08, Bogdan Tanasa <ta...@gmail.com> wrote:
> Dear all,
>
> after i am setting up an Apache Spark cluster on AWS, when i am trying to
> connect to it, the message i receive is shown below.
>
> Any suggestions would be appreciated please. Thanks a lot !
>
> ssh -i week7.pem hadoop@ec2-54-236-38-60.compute-1.amazonaws.com
> ssh: connect to host ec2-54-236-38-60.compute-1.amazonaws.com port 22:
> Connection timed out ...
>
> [image: Screenshot from 2021-03-22 09-41-11.png]
>
>
>