You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Angel Angel <ar...@gmail.com> on 2015/08/03 04:16:07 UTC
spark cluster setup
Hello Sir,
I have install the spark.
The local spark-shell is working fine.
But whenever I tried the Master configuration I got some errors.
When I run this command ;
MASTER=spark://hadoopm0:7077 spark-shell
I gets the errors likes;
15/07/27 21:17:26 INFO AppClient$ClientActor: Connecting to master
spark://hadoopm0:7077...
15/07/27 21:17:46 ERROR SparkDeploySchedulerBackend: Application has been
killed. Reason: All masters are unresponsive! Giving up.
15/07/27 21:17:46 WARN SparkDeploySchedulerBackend: Application ID is not
initialized yet.
15/07/27 21:17:46 ERROR TaskSchedulerImpl: Exiting due to error from
cluster scheduler: All masters are unresponsive! Giving up.
Also I have attached the my screenshot of Master UI.
Also i have tested using telnet command:
it shows that hadoopm0 is connected
Can you please give me some references, documentations or how to solve
this issue.
Thanks in advance.
Thanking You,
Re: spark cluster setup
Posted by Sonal Goyal <so...@gmail.com>.
Your master log files will be on the spark home folder/logs at the master
machine. Do they show an error ?
Best Regards,
Sonal
Founder, Nube Technologies <http://www.nubetech.co>
Check out Reifier at Spark Summit 2015
<https://spark-summit.org/2015/events/real-time-fuzzy-matching-with-spark-and-elastic-search/>
<http://in.linkedin.com/in/sonalgoyal>
On Mon, Aug 3, 2015 at 9:27 AM, Angel Angel <ar...@gmail.com> wrote:
> Hi,
>
> i have attached the snapshot of console.
> actually i don't know how to see the Master logs.
> still i have attache the my master web UI.
>
> and the is log file errors.
>
>
>
>
> 2015-07-23 17:00:59,977 ERROR
> org.apache.spark.scheduler.ReplayListenerBus: Malformed line: <not started>
>
>
> 2015-07-23 17:01:00,096 INFO org.eclipse.jetty.server.Server:
> jetty-8.y.z-SNAPSHOT
>
> 2015-07-23 17:01:00,138 INFO org.eclipse.jetty.server.AbstractConnector:
> Started SelectChannelConnector@0.0.0.0:18088
>
> 2015-07-23 17:01:00,138 INFO org.apache.spark.util.Utils: Successfully
> started service on port 18088.
>
> 2015-07-23 17:01:00,140 INFO
> org.apache.spark.deploy.history.HistoryServer: Started HistoryServer at
> http://hadoopm0:18088
>
> 2015-07-24 11:36:18,148 INFO org.apache.spark.SecurityManager: Changing
> view acls to: spark
>
> 2015-07-24 11:36:18,148 INFO org.apache.spark.SecurityManager: Changing
> modify acls to: spark
>
> 2015-07-24 11:36:18,148 INFO org.apache.spark.SecurityManager:
> SecurityManager: authentication disabled; ui acls disabled; users with view
> permissions: Set(spark); users with modify permissions: Set(spark)
>
> 2015-07-24 11:36:18,367 INFO org.apache.spark.SecurityManager: Changing
> acls enabled to: false
>
> 2015-07-24 11:36:18,367 INFO org.apache.spark.SecurityManager: Changing
> admin acls to:
>
> 2015-07-24 11:36:18,368 INFO org.apache.spark.SecurityManager: Changing
> view acls to: root
>
>
> Thanks.
>
>
> On Mon, Aug 3, 2015 at 11:52 AM, Sonal Goyal <so...@gmail.com>
> wrote:
>
>> What do the master logs show?
>>
>> Best Regards,
>> Sonal
>> Founder, Nube Technologies
>> <http://t.sidekickopen13.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJW7t5XZs1pNkJdVdDLZW1q7zBxW64k9XR56dLFLf58_ZT802?t=http%3A%2F%2Fwww.nubetech.co%2F&si=5462006004973568&pi=903294d1-e4a2-4926-cf03-b51cc168cfc1>
>>
>> Check out Reifier at Spark Summit 2015
>> <http://t.sidekickopen13.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJW7t5XZs1pNkJdVdDLZW1q7zBxW64k9XR56dLFLf58_ZT802?t=https%3A%2F%2Fspark-summit.org%2F2015%2Fevents%2Freal-time-fuzzy-matching-with-spark-and-elastic-search%2F&si=5462006004973568&pi=903294d1-e4a2-4926-cf03-b51cc168cfc1>
>>
>>
>> <http://t.sidekickopen13.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJW7t5XZs1pNkJdVdDLZW1q7zBxW64k9XR56dLFLf58_ZT802?t=http%3A%2F%2Fin.linkedin.com%2Fin%2Fsonalgoyal&si=5462006004973568&pi=903294d1-e4a2-4926-cf03-b51cc168cfc1>
>>
>>
>>
>> On Mon, Aug 3, 2015 at 7:46 AM, Angel Angel <ar...@gmail.com>
>> wrote:
>>
>>> Hello Sir,
>>>
>>> I have install the spark.
>>>
>>>
>>>
>>> The local spark-shell is working fine.
>>>
>>>
>>>
>>> But whenever I tried the Master configuration I got some errors.
>>>
>>>
>>>
>>> When I run this command ;
>>>
>>> MASTER=spark://hadoopm0:7077 spark-shell
>>>
>>>
>>>
>>> I gets the errors likes;
>>>
>>>
>>>
>>> 15/07/27 21:17:26 INFO AppClient$ClientActor: Connecting to master
>>> spark://hadoopm0:7077...
>>>
>>> 15/07/27 21:17:46 ERROR SparkDeploySchedulerBackend: Application has
>>> been killed. Reason: All masters are unresponsive! Giving up.
>>>
>>> 15/07/27 21:17:46 WARN SparkDeploySchedulerBackend: Application ID is
>>> not initialized yet.
>>>
>>> 15/07/27 21:17:46 ERROR TaskSchedulerImpl: Exiting due to error from
>>> cluster scheduler: All masters are unresponsive! Giving up.
>>>
>>>
>>>
>>> Also I have attached the my screenshot of Master UI.
>>>
>>>
>>> Also i have tested using telnet command:
>>>
>>>
>>> it shows that hadoopm0 is connected
>>>
>>>
>>>
>>> Can you please give me some references, documentations or how to solve
>>> this issue.
>>>
>>> Thanks in advance.
>>>
>>> Thanking You,
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: user-help@spark.apache.org
>>>
>>
>>
>
Re: spark cluster setup
Posted by Sonal Goyal <so...@gmail.com>.
What do the master logs show?
Best Regards,
Sonal
Founder, Nube Technologies
<http://t.sidekickopen13.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJW7t5XZs1pNkJdVdDLZW1q7zBxW64k9XR56dLFLf58_ZT802?t=http%3A%2F%2Fwww.nubetech.co%2F&si=5462006004973568&pi=903294d1-e4a2-4926-cf03-b51cc168cfc1>
Check out Reifier at Spark Summit 2015
<http://t.sidekickopen13.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJW7t5XZs1pNkJdVdDLZW1q7zBxW64k9XR56dLFLf58_ZT802?t=https%3A%2F%2Fspark-summit.org%2F2015%2Fevents%2Freal-time-fuzzy-matching-with-spark-and-elastic-search%2F&si=5462006004973568&pi=903294d1-e4a2-4926-cf03-b51cc168cfc1>
<http://t.sidekickopen13.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJW7t5XZs1pNkJdVdDLZW1q7zBxW64k9XR56dLFLf58_ZT802?t=http%3A%2F%2Fin.linkedin.com%2Fin%2Fsonalgoyal&si=5462006004973568&pi=903294d1-e4a2-4926-cf03-b51cc168cfc1>
On Mon, Aug 3, 2015 at 7:46 AM, Angel Angel <ar...@gmail.com> wrote:
> Hello Sir,
>
> I have install the spark.
>
>
>
> The local spark-shell is working fine.
>
>
>
> But whenever I tried the Master configuration I got some errors.
>
>
>
> When I run this command ;
>
> MASTER=spark://hadoopm0:7077 spark-shell
>
>
>
> I gets the errors likes;
>
>
>
> 15/07/27 21:17:26 INFO AppClient$ClientActor: Connecting to master
> spark://hadoopm0:7077...
>
> 15/07/27 21:17:46 ERROR SparkDeploySchedulerBackend: Application has been
> killed. Reason: All masters are unresponsive! Giving up.
>
> 15/07/27 21:17:46 WARN SparkDeploySchedulerBackend: Application ID is not
> initialized yet.
>
> 15/07/27 21:17:46 ERROR TaskSchedulerImpl: Exiting due to error from
> cluster scheduler: All masters are unresponsive! Giving up.
>
>
>
> Also I have attached the my screenshot of Master UI.
>
>
> Also i have tested using telnet command:
>
>
> it shows that hadoopm0 is connected
>
>
>
> Can you please give me some references, documentations or how to solve
> this issue.
>
> Thanks in advance.
>
> Thanking You,
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
Re: spark cluster setup
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Are you sitting behind a firewall and accessing a remote master machine? In
that case, have a look at this
http://spark.apache.org/docs/latest/configuration.html#networking, you
might want to fix few properties like spark.driver.host, spark.driver.host
etc.
Thanks
Best Regards
On Mon, Aug 3, 2015 at 7:46 AM, Angel Angel <ar...@gmail.com> wrote:
> Hello Sir,
>
> I have install the spark.
>
>
>
> The local spark-shell is working fine.
>
>
>
> But whenever I tried the Master configuration I got some errors.
>
>
>
> When I run this command ;
>
> MASTER=spark://hadoopm0:7077 spark-shell
>
>
>
> I gets the errors likes;
>
>
>
> 15/07/27 21:17:26 INFO AppClient$ClientActor: Connecting to master
> spark://hadoopm0:7077...
>
> 15/07/27 21:17:46 ERROR SparkDeploySchedulerBackend: Application has been
> killed. Reason: All masters are unresponsive! Giving up.
>
> 15/07/27 21:17:46 WARN SparkDeploySchedulerBackend: Application ID is not
> initialized yet.
>
> 15/07/27 21:17:46 ERROR TaskSchedulerImpl: Exiting due to error from
> cluster scheduler: All masters are unresponsive! Giving up.
>
>
>
> Also I have attached the my screenshot of Master UI.
>
>
> Also i have tested using telnet command:
>
>
> it shows that hadoopm0 is connected
>
>
>
> Can you please give me some references, documentations or how to solve
> this issue.
>
> Thanks in advance.
>
> Thanking You,
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>