You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jong Wook Kim <jo...@nyu.edu> on 2015/11/16 03:59:32 UTC
Spark-shell connecting to Mesos stuck at sched.cpp
I'm having problem connecting my spark app to a Mesos cluster; any help on
the below question would be appreciated.
http://stackoverflow.com/questions/33727154/spark-shell-connecting-to-mesos-stuck-at-sched-cpp
Thanks,
Jong Wook
Re: Spark-shell connecting to Mesos stuck at sched.cpp
Posted by Aaron <aa...@gmail.com>.
Found this thread that talked about it to help understand it better:
https://mail-archives.apache.org/mod_mbox/mesos-user/201507.mbox/%3CCAJQ68qf9pEJgNWOmasM2DQchYaxPcAovnFKFgGgxXpZJ2JOAfQ@mail.gmail.com%3E
>
> When you run Spark on Mesos it needs to run
>
> spark driver
> mesos scheduler
>
> and both need to be visible to outside world on public iface IP
>
> you need to tell Spark and Mesos on which interface to bind - by default
> they resolve node hostname to ip - this is loopback address in your case
>
> Possible solutions - on slave node with public IP 192.168.56.50
>
> 1. Set
>
> export LIBPROCESS_IP=192.168.56.50
> export SPARK_LOCAL_IP=192.168.56.50
>
> 2. Ensure your hostname resolves to public iface IP - (for testing) edit
> /etc/hosts to resolve your domain name to 192.168.56.50
> 3. Set correct hostname/ip in mesos configuration - see Nikolaos answer
>
Cheers,
Aaron
On Mon, Nov 16, 2015 at 9:37 PM, Jo Voordeckers
<jo...@gmail.com> wrote:
> I've seen this issue when the mesos cluster couldn't figure out my IP
> address correctly, have you tried setting the ENV var with your IP address
> when launching spark or mesos cluster dispatcher like:
>
> LIBPROCESS_IP="172.16.0.180"
>
>
> - Jo Voordeckers
>
>
> On Sun, Nov 15, 2015 at 6:59 PM, Jong Wook Kim <jo...@nyu.edu> wrote:
>>
>> I'm having problem connecting my spark app to a Mesos cluster; any help on
>> the below question would be appreciated.
>>
>>
>> http://stackoverflow.com/questions/33727154/spark-shell-connecting-to-mesos-stuck-at-sched-cpp
>>
>> Thanks,
>> Jong Wook
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Spark-shell connecting to Mesos stuck at sched.cpp
Posted by Jo Voordeckers <jo...@gmail.com>.
I've seen this issue when the mesos cluster couldn't figure out my IP
address correctly, have you tried setting the ENV var with your IP address
when launching spark or mesos cluster dispatcher like:
LIBPROCESS_IP="172.16.0.180"
- Jo Voordeckers
On Sun, Nov 15, 2015 at 6:59 PM, Jong Wook Kim <jo...@nyu.edu> wrote:
> I'm having problem connecting my spark app to a Mesos cluster; any help on
> the below question would be appreciated.
>
>
> http://stackoverflow.com/questions/33727154/spark-shell-connecting-to-mesos-stuck-at-sched-cpp
>
> Thanks,
> Jong Wook
>