You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Xingwei Yang <ha...@gmail.com> on 2014/12/04 23:26:57 UTC
Unable to run applications on clusters on EC2
I think it is related to my previous questions, but I separate them. In my
previous question, I could not connect to WebUI even though I could log
into the cluster without any problem.
Also, I tried lynx localhost:8080 and I could get the information about the
cluster;
I could also user spark-submit to submit job locally by setting master to
localhost
However, I could not submit the job to the cluster master and I get the
error like this:
14/12/04 22:14:39 WARN scheduler.TaskSchedulerImpl: Initial job has not
accepted any resources; check your cluster UI to ensure that workers are
registered and have sufficient memory
14/12/04 22:14:42 INFO client.AppClient$ClientActor: Connecting to master
spark://ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070...
14/12/04 22:14:42 WARN client.AppClient$ClientActor: Could not connect to
akka.tcp://
sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://
sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070]
14/12/04 22:14:42 WARN client.AppClient$ClientActor: Could not connect to
akka.tcp://
sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://
sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070]
14/12/04 22:14:42 WARN client.AppClient$ClientActor: Could not connect to
akka.tcp://
sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://
sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070]
14/12/04 22:14:42 WARN client.AppClient$ClientActor: Could not connect to
akka.tcp://
sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://
sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070]
Please let me know if you any any clue about it. Thanks a lot.
--
Sincerely Yours
Xingwei Yang
https://sites.google.com/site/xingweiyang1223/
Re: Unable to run applications on clusters on EC2
Posted by Andrew Or <an...@databricks.com>.
Hey, the default port is 7077. Not sure if you actually meant to put 7070.
As a rule of thumb, you can go to the Master web UI and copy and paste the
URL at the top left corner. That almost always works unless your cluster
has a weird proxy set up.
2014-12-04 14:26 GMT-08:00 Xingwei Yang <ha...@gmail.com>:
> I think it is related to my previous questions, but I separate them. In my
> previous question, I could not connect to WebUI even though I could log
> into the cluster without any problem.
>
> Also, I tried lynx localhost:8080 and I could get the information about
> the cluster;
>
> I could also user spark-submit to submit job locally by setting master to
> localhost
>
>
> However, I could not submit the job to the cluster master and I get the
> error like this:
>
> 14/12/04 22:14:39 WARN scheduler.TaskSchedulerImpl: Initial job has not
> accepted any resources; check your cluster UI to ensure that workers are
> registered and have sufficient memory
> 14/12/04 22:14:42 INFO client.AppClient$ClientActor: Connecting to master
> spark://ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070...
> 14/12/04 22:14:42 WARN client.AppClient$ClientActor: Could not connect to
> akka.tcp://
> sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070:
> akka.remote.EndpointAssociationException: Association failed with
> [akka.tcp://
> sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070]
> 14/12/04 22:14:42 WARN client.AppClient$ClientActor: Could not connect to
> akka.tcp://
> sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070:
> akka.remote.EndpointAssociationException: Association failed with
> [akka.tcp://
> sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070]
> 14/12/04 22:14:42 WARN client.AppClient$ClientActor: Could not connect to
> akka.tcp://
> sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070:
> akka.remote.EndpointAssociationException: Association failed with
> [akka.tcp://
> sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070]
> 14/12/04 22:14:42 WARN client.AppClient$ClientActor: Could not connect to
> akka.tcp://
> sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070:
> akka.remote.EndpointAssociationException: Association failed with
> [akka.tcp://
> sparkMaster@ec2-54-149-92-187.us-west-2.compute.amazonaws.com:7070]
>
>
> Please let me know if you any any clue about it. Thanks a lot.
>
> --
> Sincerely Yours
> Xingwei Yang
> https://sites.google.com/site/xingweiyang1223/
>
>