You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Haripriya Ayyalasomayajula <ah...@gmail.com> on 2015/07/28 20:16:35 UTC
Problems connecting with Mesos Master
Hi all,
I am trying to use Spark 1.4.1 with Mesos 0.23.0.
When I try to start my spark-shell, it gives me the following warning :
**************************************************
Scheduler driver bound to loopback interface! Cannot communicate with
remote master(s). You might want to set 'LIBPROCESS_IP' environment
variable to use a routable IP address.
---
Spark-shell works fine on the node where I run master, but if I start
running on any of the other slave nodes it gives me the following error:
E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on fd=6:
Transport endpoint is not connected [107]
E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6:
Transport endpoint is not connected [107]
I have the following configs:
- zookeeper configured to the mesos master
- /etc/mesos/zk on all nodes pointing to mesos master ip.
I am not sure if I have to set the ip flag and where I have to set the --ip
flag?
--
Regards,
Haripriya Ayyalasomayajula
Re: Problems connecting with Mesos Master
Posted by Ondrej Smola <on...@gmail.com>.
This means registration cannot be completed - usually caused by mesos
scheduler (on your slave) reporting wrong ip/hostname - you should see some
info in master UI on port 5050 - tab frameworks. Look at host your slave
reported on master UI slave tab.
Make sure you correcly setup files in step 3 in my previous answer and
restart mesos-slave.
2015-07-29 18:37 GMT+02:00 Haripriya Ayyalasomayajula <
aharipriya92@gmail.com>:
> Thanks. It does not complain about LIBPROCESS any more. It is able to
> connect to the master, but hangs before the framework can register.
>
> Can you please help me understand where I am going wrong?
>
> I0729 11:19:55.893210 46270 detector.cpp:481] A new leading master
> (UPID=master@my_ip:5050) is detected
>
> I0729 11:19:55.892369 46272 sched.cpp:254] New master detected at master@my
>
> _ip:5050
>
> I0729 11:19:55.893728 46272 sched.cpp:264] No credentials provided.
> Attempting to register without authentication
>
> On Tue, Jul 28, 2015 at 10:35 PM, Ondrej Smola <on...@gmail.com>
> wrote:
>
>> Hi Haripiya,
>>
>> When you run Spark on Mesos it needs to run
>>
>> spark driver
>> mesos scheduler
>>
>> and both need to be visible to outside world on public iface IP
>>
>> you need to tell Spark and Mesos on which interface to bind - by default
>> they resolve node hostname to ip - this is loopback address in your case
>>
>> Possible solutions - on slave node with public IP 192.168.56.50
>>
>> 1. Set
>>
>> export LIBPROCESS_IP=192.168.56.50
>> export SPARK_LOCAL_IP=192.168.56.50
>>
>> 2. Ensure your hostname resolves to public iface IP - (for testing) edit
>> /etc/hosts to resolve your domain name to 192.168.56.50
>> 3. Set correct hostname/ip in mesos configuration - see Nikolaos answer
>>
>>
>>
>>
>> 2015-07-29 1:04 GMT+02:00 Haripriya Ayyalasomayajula <
>> aharipriya92@gmail.com>:
>>
>>> It quits before it writes any logs, when I look at the directory for
>>> logs, it has empty files!
>>>
>>> On Tue, Jul 28, 2015 at 2:34 PM, Vinod Kone <vi...@gmail.com> wrote:
>>>
>>>> can you paste the logs?
>>>>
>>>> On Tue, Jul 28, 2015 at 2:31 PM, Haripriya Ayyalasomayajula <
>>>> aharipriya92@gmail.com> wrote:
>>>>
>>>>> Well, when I try doing that I get this error:
>>>>>
>>>>> Failed to initialize: Failed to bind on scheduler_ip_address Cannot
>>>>> assign requested address: Cannot assign requested address [99]
>>>>>
>>>>> When I do a
>>>>>
>>>>> ps -ef | grep mesos
>>>>>
>>>>> on both my master and slave nodes, it works fine. And, I am also able
>>>>> to ping both the nodes from each other - they are reachable to one another.
>>>>>
>>>>> On Tue, Jul 28, 2015 at 2:18 PM, Vinod Kone <vi...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> LIBPROCESS_IP is the IP address that you want the scheduler (driver)
>>>>>> to bind to. It has nothing to do with the ZooKeeper address.
>>>>>>
>>>>>> In other words, do
>>>>>>
>>>>>> export LIBPROCESS_IP=scheduler_ip_address
>>>>>>
>>>>>>
>>>>>> On Tue, Jul 28, 2015 at 2:00 PM, Haripriya Ayyalasomayajula <
>>>>>> aharipriya92@gmail.com> wrote:
>>>>>>
>>>>>>> I am trying to do this
>>>>>>>
>>>>>>> export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos
>>>>>>>
>>>>>>> ./bin/spark-shell
>>>>>>>
>>>>>>> It gives me this error and aborts
>>>>>>>
>>>>>>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>>>>>>>
>>>>>>> F0728 15:43:39.361445 13209 process.cpp:847] Parsing
>>>>>>> LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP
>>>>>>>
>>>>>>> On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <ti...@mesosphere.io> wrote:
>>>>>>>
>>>>>>>> spark-env.sh works as it will be called by
>>>>>>>> spark-submit/spark-shell, or you can just set it before you call
>>>>>>>> spark-shell yourself.
>>>>>>>>
>>>>>>>> Tim
>>>>>>>>
>>>>>>>> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula <
>>>>>>>> aharipriya92@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Where can I set the libprocess_ip env variable? spark_env.sh?
>>>>>>>>> Thats the only place I can think of. Can you please point me to any related
>>>>>>>>> documentation?
>>>>>>>>>
>>>>>>>>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <
>>>>>>>>> nikolaos.ballas@nexusgroup.com> wrote:
>>>>>>>>>
>>>>>>>>>> If you are not using any dns like service under
>>>>>>>>>> /etc/mesos-master/ create two files called ip and hostname and put the ip
>>>>>>>>>> of the eth interface.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Sent from my Samsung device
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -------- Original message --------
>>>>>>>>>> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
>>>>>>>>>> Date: 28/07/2015 20:18 (GMT+01:00)
>>>>>>>>>> To: user@mesos.apache.org
>>>>>>>>>> Subject: Problems connecting with Mesos Master
>>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>>>>>>>>>>
>>>>>>>>>> When I try to start my spark-shell, it gives me the following
>>>>>>>>>> warning :
>>>>>>>>>>
>>>>>>>>>> **************************************************
>>>>>>>>>>
>>>>>>>>>> Scheduler driver bound to loopback interface! Cannot communicate
>>>>>>>>>> with remote master(s). You might want to set 'LIBPROCESS_IP' environment
>>>>>>>>>> variable to use a routable IP address.
>>>>>>>>>> ---
>>>>>>>>>>
>>>>>>>>>> Spark-shell works fine on the node where I run master, but if I
>>>>>>>>>> start running on any of the other slave nodes it gives me the following
>>>>>>>>>> error:
>>>>>>>>>>
>>>>>>>>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on
>>>>>>>>>> fd=6: Transport endpoint is not connected [107]
>>>>>>>>>>
>>>>>>>>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on
>>>>>>>>>> fd=6: Transport endpoint is not connected [107]
>>>>>>>>>>
>>>>>>>>>> I have the following configs:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - zookeeper configured to the mesos master
>>>>>>>>>> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>>>>>>>>>>
>>>>>>>>>> I am not sure if I have to set the ip flag and where I have to
>>>>>>>>>> set the --ip flag?
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Regards,
>>>>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Regards,
>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Haripriya Ayyalasomayajula
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Haripriya Ayyalasomayajula
>>>
>>>
>>
>
>
> --
> Regards,
> Haripriya Ayyalasomayajula
>
>
Re: Problems connecting with Mesos Master
Posted by Haripriya Ayyalasomayajula <ah...@gmail.com>.
Hi all,
Thank you! It works fine now! The problem was with my /etc/hosts file and
as you suggested it had problem with resolution of hostname. When I added
LIBPROCESS_IP in my spark_env.sh and also the SPARK_LOCAL_IP, re-configured
/etc/hosts file with correct set of ips, it works now!
Thanks a lot for the help!
On Wed, Jul 29, 2015 at 10:04 AM, Shafay Latif <sl...@apple.com> wrote:
> Are you enabling authenticate_slaves flag on your master? If so, are *cyrus-sasl-devel cyrus-sasl-md5
> rpms installed? On the masters and service restarted?*
> *Have you cleaned up previous meta data on the slaves?*
> rm -vf /tmp/mesos/meta/slaves/latest
>
>
>
>
> Shafay Latif
>
> On Jul 29, 2015, at 9:37 AM, Haripriya Ayyalasomayajula <
> aharipriya92@gmail.com> wrote:
>
> Thanks. It does not complain about LIBPROCESS any more. It is able to
> connect to the master, but hangs before the framework can register.
>
> Can you please help me understand where I am going wrong?
>
> I0729 11:19:55.893210 46270 detector.cpp:481] A new leading master
> (UPID=master@my_ip:5050) is detected
>
> I0729 11:19:55.892369 46272 sched.cpp:254] New master detected at master@my
>
> _ip:5050
>
> I0729 11:19:55.893728 46272 sched.cpp:264] No credentials provided.
> Attempting to register without authentication
>
> On Tue, Jul 28, 2015 at 10:35 PM, Ondrej Smola <on...@gmail.com>
> wrote:
>
>> Hi Haripiya,
>>
>> When you run Spark on Mesos it needs to run
>>
>> spark driver
>> mesos scheduler
>>
>> and both need to be visible to outside world on public iface IP
>>
>> you need to tell Spark and Mesos on which interface to bind - by default
>> they resolve node hostname to ip - this is loopback address in your case
>>
>> Possible solutions - on slave node with public IP 192.168.56.50
>>
>> 1. Set
>>
>> export LIBPROCESS_IP=192.168.56.50
>> export SPARK_LOCAL_IP=192.168.56.50
>>
>> 2. Ensure your hostname resolves to public iface IP - (for testing) edit
>> /etc/hosts to resolve your domain name to 192.168.56.50
>> 3. Set correct hostname/ip in mesos configuration - see Nikolaos answer
>>
>>
>>
>>
>> 2015-07-29 1:04 GMT+02:00 Haripriya Ayyalasomayajula <
>> aharipriya92@gmail.com>:
>>
>>> It quits before it writes any logs, when I look at the directory for
>>> logs, it has empty files!
>>>
>>> On Tue, Jul 28, 2015 at 2:34 PM, Vinod Kone <vi...@gmail.com> wrote:
>>>
>>>> can you paste the logs?
>>>>
>>>> On Tue, Jul 28, 2015 at 2:31 PM, Haripriya Ayyalasomayajula <
>>>> aharipriya92@gmail.com> wrote:
>>>>
>>>>> Well, when I try doing that I get this error:
>>>>>
>>>>> Failed to initialize: Failed to bind on scheduler_ip_address Cannot
>>>>> assign requested address: Cannot assign requested address [99]
>>>>>
>>>>> When I do a
>>>>>
>>>>> ps -ef | grep mesos
>>>>>
>>>>> on both my master and slave nodes, it works fine. And, I am also able
>>>>> to ping both the nodes from each other - they are reachable to one another.
>>>>>
>>>>> On Tue, Jul 28, 2015 at 2:18 PM, Vinod Kone <vi...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> LIBPROCESS_IP is the IP address that you want the scheduler (driver)
>>>>>> to bind to. It has nothing to do with the ZooKeeper address.
>>>>>>
>>>>>> In other words, do
>>>>>>
>>>>>> export LIBPROCESS_IP=scheduler_ip_address
>>>>>>
>>>>>>
>>>>>> On Tue, Jul 28, 2015 at 2:00 PM, Haripriya Ayyalasomayajula <
>>>>>> aharipriya92@gmail.com> wrote:
>>>>>>
>>>>>>> I am trying to do this
>>>>>>>
>>>>>>> export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos
>>>>>>>
>>>>>>> ./bin/spark-shell
>>>>>>>
>>>>>>> It gives me this error and aborts
>>>>>>>
>>>>>>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>>>>>>>
>>>>>>> F0728 15:43:39.361445 13209 process.cpp:847] Parsing
>>>>>>> LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP
>>>>>>>
>>>>>>> On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <ti...@mesosphere.io> wrote:
>>>>>>>
>>>>>>>> spark-env.sh works as it will be called by
>>>>>>>> spark-submit/spark-shell, or you can just set it before you call
>>>>>>>> spark-shell yourself.
>>>>>>>>
>>>>>>>> Tim
>>>>>>>>
>>>>>>>> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula <
>>>>>>>> aharipriya92@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Where can I set the libprocess_ip env variable? spark_env.sh?
>>>>>>>>> Thats the only place I can think of. Can you please point me to any related
>>>>>>>>> documentation?
>>>>>>>>>
>>>>>>>>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <
>>>>>>>>> nikolaos.ballas@nexusgroup.com> wrote:
>>>>>>>>>
>>>>>>>>>> If you are not using any dns like service under
>>>>>>>>>> /etc/mesos-master/ create two files called ip and hostname and put the ip
>>>>>>>>>> of the eth interface.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Sent from my Samsung device
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -------- Original message --------
>>>>>>>>>> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
>>>>>>>>>> Date: 28/07/2015 20:18 (GMT+01:00)
>>>>>>>>>> To: user@mesos.apache.org
>>>>>>>>>> Subject: Problems connecting with Mesos Master
>>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>>>>>>>>>>
>>>>>>>>>> When I try to start my spark-shell, it gives me the following
>>>>>>>>>> warning :
>>>>>>>>>>
>>>>>>>>>> **************************************************
>>>>>>>>>>
>>>>>>>>>> Scheduler driver bound to loopback interface! Cannot communicate
>>>>>>>>>> with remote master(s). You might want to set 'LIBPROCESS_IP' environment
>>>>>>>>>> variable to use a routable IP address.
>>>>>>>>>> ---
>>>>>>>>>>
>>>>>>>>>> Spark-shell works fine on the node where I run master, but if I
>>>>>>>>>> start running on any of the other slave nodes it gives me the following
>>>>>>>>>> error:
>>>>>>>>>>
>>>>>>>>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on
>>>>>>>>>> fd=6: Transport endpoint is not connected [107]
>>>>>>>>>>
>>>>>>>>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on
>>>>>>>>>> fd=6: Transport endpoint is not connected [107]
>>>>>>>>>>
>>>>>>>>>> I have the following configs:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - zookeeper configured to the mesos master
>>>>>>>>>> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>>>>>>>>>>
>>>>>>>>>> I am not sure if I have to set the ip flag and where I have to
>>>>>>>>>> set the --ip flag?
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Regards,
>>>>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Regards,
>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Haripriya Ayyalasomayajula
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Haripriya Ayyalasomayajula
>>>
>>>
>>
>
>
> --
> Regards,
> Haripriya Ayyalasomayajula
>
>
--
Regards,
Haripriya Ayyalasomayajula
Re: Problems connecting with Mesos Master
Posted by Shafay Latif <sl...@apple.com>.
Are you enabling authenticate_slaves flag on your master? If so, are cyrus-sasl-devel cyrus-sasl-md5 rpms installed? On the masters and service restarted?
Have you cleaned up previous meta data on the slaves?
rm -vf /tmp/mesos/meta/slaves/latest
Shafay Latif
On Jul 29, 2015, at 9:37 AM, Haripriya Ayyalasomayajula <ah...@gmail.com> wrote:
Thanks. It does not complain about LIBPROCESS any more. It is able to connect to the master, but hangs before the framework can register.
Can you please help me understand where I am going wrong?
I0729 11:19:55.893210 46270 detector.cpp:481] A new leading master (UPID=master@my_ip:5050) is detected
I0729 11:19:55.892369 46272 sched.cpp:254] New master detected at master@my
_ip:5050
I0729 11:19:55.893728 46272 sched.cpp:264] No credentials provided. Attempting to register without authentication
> On Tue, Jul 28, 2015 at 10:35 PM, Ondrej Smola <on...@gmail.com> wrote:
> Hi Haripiya,
>
> When you run Spark on Mesos it needs to run
>
> spark driver
> mesos scheduler
>
> and both need to be visible to outside world on public iface IP
>
> you need to tell Spark and Mesos on which interface to bind - by default they resolve node hostname to ip - this is loopback address in your case
>
> Possible solutions - on slave node with public IP 192.168.56.50
>
> 1. Set
>
> export LIBPROCESS_IP=192.168.56.50
> export SPARK_LOCAL_IP=192.168.56.50
>
> 2. Ensure your hostname resolves to public iface IP - (for testing) edit /etc/hosts to resolve your domain name to 192.168.56.50
> 3. Set correct hostname/ip in mesos configuration - see Nikolaos answer
>
>
>
>
> 2015-07-29 1:04 GMT+02:00 Haripriya Ayyalasomayajula <ah...@gmail.com>:
>> It quits before it writes any logs, when I look at the directory for logs, it has empty files!
>>
>>> On Tue, Jul 28, 2015 at 2:34 PM, Vinod Kone <vi...@gmail.com> wrote:
>>> can you paste the logs?
>>>
>>>> On Tue, Jul 28, 2015 at 2:31 PM, Haripriya Ayyalasomayajula <ah...@gmail.com> wrote:
>>>> Well, when I try doing that I get this error:
>>>>
>>>> Failed to initialize: Failed to bind on scheduler_ip_address Cannot assign requested address: Cannot assign requested address [99]
>>>>
>>>> When I do a
>>>>
>>>> ps -ef | grep mesos
>>>>
>>>> on both my master and slave nodes, it works fine. And, I am also able to ping both the nodes from each other - they are reachable to one another.
>>>>
>>>> On Tue, Jul 28, 2015 at 2:18 PM, Vinod Kone <vi...@gmail.com> wrote:
>>>>> LIBPROCESS_IP is the IP address that you want the scheduler (driver) to bind to. It has nothing to do with the ZooKeeper address.
>>>>>
>>>>> In other words, do
>>>>>
>>>>> export LIBPROCESS_IP=scheduler_ip_address
>>>>>
>>>>>
>>>>>> On Tue, Jul 28, 2015 at 2:00 PM, Haripriya Ayyalasomayajula <ah...@gmail.com> wrote:
>>>>>> I am trying to do this
>>>>>>
>>>>>> export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos
>>>>>> ./bin/spark-shell
>>>>>>
>>>>>> It gives me this error and aborts
>>>>>>
>>>>>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>>>>>>
>>>>>>
>>>>>> F0728 15:43:39.361445 13209 process.cpp:847] Parsing LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP
>>>>>>
>>>>>>
>>>>>>> On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <ti...@mesosphere.io> wrote:
>>>>>>> spark-env.sh works as it will be called by spark-submit/spark-shell, or you can just set it before you call spark-shell yourself.
>>>>>>>
>>>>>>> Tim
>>>>>>>
>>>>>>>> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula <ah...@gmail.com> wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Where can I set the libprocess_ip env variable? spark_env.sh? Thats the only place I can think of. Can you please point me to any related documentation?
>>>>>>>>
>>>>>>>>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <ni...@nexusgroup.com> wrote:
>>>>>>>>> If you are not using any dns like service under /etc/mesos-master/ create two files called ip and hostname and put the ip of the eth interface.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Sent from my Samsung device
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -------- Original message --------
>>>>>>>>> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
>>>>>>>>> Date: 28/07/2015 20:18 (GMT+01:00)
>>>>>>>>> To: user@mesos.apache.org
>>>>>>>>> Subject: Problems connecting with Mesos Master
>>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>>>>>>>>>
>>>>>>>>> When I try to start my spark-shell, it gives me the following warning :
>>>>>>>>> **************************************************
>>>>>>>>>
>>>>>>>>> Scheduler driver bound to loopback interface! Cannot communicate with remote master(s). You might want to set 'LIBPROCESS_IP' environment variable to use a routable IP address.
>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>> Spark-shell works fine on the node where I run master, but if I start running on any of the other slave nodes it gives me the following error:
>>>>>>>>>
>>>>>>>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on fd=6: Transport endpoint is not connected [107]
>>>>>>>>>
>>>>>>>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6: Transport endpoint is not connected [107]
>>>>>>>>>
>>>>>>>>> I have the following configs:
>>>>>>>>>
>>>>>>>>> zookeeper configured to the mesos master
>>>>>>>>> /etc/mesos/zk on all nodes pointing to mesos master ip.
>>>>>>>>> I am not sure if I have to set the ip flag and where I have to set the --ip flag?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Regards,
>>>>>>>> Haripriya Ayyalasomayajula
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Haripriya Ayyalasomayajula
>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Haripriya Ayyalasomayajula
>>
>>
>>
>> --
>> Regards,
>> Haripriya Ayyalasomayajula
--
Regards,
Haripriya Ayyalasomayajula
Re: Problems connecting with Mesos Master
Posted by Haripriya Ayyalasomayajula <ah...@gmail.com>.
Thanks. It does not complain about LIBPROCESS any more. It is able to
connect to the master, but hangs before the framework can register.
Can you please help me understand where I am going wrong?
I0729 11:19:55.893210 46270 detector.cpp:481] A new leading master
(UPID=master@my_ip:5050) is detected
I0729 11:19:55.892369 46272 sched.cpp:254] New master detected at master@my
_ip:5050
I0729 11:19:55.893728 46272 sched.cpp:264] No credentials provided.
Attempting to register without authentication
On Tue, Jul 28, 2015 at 10:35 PM, Ondrej Smola <on...@gmail.com>
wrote:
> Hi Haripiya,
>
> When you run Spark on Mesos it needs to run
>
> spark driver
> mesos scheduler
>
> and both need to be visible to outside world on public iface IP
>
> you need to tell Spark and Mesos on which interface to bind - by default
> they resolve node hostname to ip - this is loopback address in your case
>
> Possible solutions - on slave node with public IP 192.168.56.50
>
> 1. Set
>
> export LIBPROCESS_IP=192.168.56.50
> export SPARK_LOCAL_IP=192.168.56.50
>
> 2. Ensure your hostname resolves to public iface IP - (for testing) edit
> /etc/hosts to resolve your domain name to 192.168.56.50
> 3. Set correct hostname/ip in mesos configuration - see Nikolaos answer
>
>
>
>
> 2015-07-29 1:04 GMT+02:00 Haripriya Ayyalasomayajula <
> aharipriya92@gmail.com>:
>
>> It quits before it writes any logs, when I look at the directory for
>> logs, it has empty files!
>>
>> On Tue, Jul 28, 2015 at 2:34 PM, Vinod Kone <vi...@gmail.com> wrote:
>>
>>> can you paste the logs?
>>>
>>> On Tue, Jul 28, 2015 at 2:31 PM, Haripriya Ayyalasomayajula <
>>> aharipriya92@gmail.com> wrote:
>>>
>>>> Well, when I try doing that I get this error:
>>>>
>>>> Failed to initialize: Failed to bind on scheduler_ip_address Cannot
>>>> assign requested address: Cannot assign requested address [99]
>>>>
>>>> When I do a
>>>>
>>>> ps -ef | grep mesos
>>>>
>>>> on both my master and slave nodes, it works fine. And, I am also able
>>>> to ping both the nodes from each other - they are reachable to one another.
>>>>
>>>> On Tue, Jul 28, 2015 at 2:18 PM, Vinod Kone <vi...@gmail.com>
>>>> wrote:
>>>>
>>>>> LIBPROCESS_IP is the IP address that you want the scheduler (driver)
>>>>> to bind to. It has nothing to do with the ZooKeeper address.
>>>>>
>>>>> In other words, do
>>>>>
>>>>> export LIBPROCESS_IP=scheduler_ip_address
>>>>>
>>>>>
>>>>> On Tue, Jul 28, 2015 at 2:00 PM, Haripriya Ayyalasomayajula <
>>>>> aharipriya92@gmail.com> wrote:
>>>>>
>>>>>> I am trying to do this
>>>>>>
>>>>>> export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos
>>>>>>
>>>>>> ./bin/spark-shell
>>>>>>
>>>>>> It gives me this error and aborts
>>>>>>
>>>>>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>>>>>>
>>>>>> F0728 15:43:39.361445 13209 process.cpp:847] Parsing
>>>>>> LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP
>>>>>>
>>>>>> On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <ti...@mesosphere.io> wrote:
>>>>>>
>>>>>>> spark-env.sh works as it will be called by spark-submit/spark-shell,
>>>>>>> or you can just set it before you call spark-shell yourself.
>>>>>>>
>>>>>>> Tim
>>>>>>>
>>>>>>> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula <
>>>>>>> aharipriya92@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Where can I set the libprocess_ip env variable? spark_env.sh? Thats
>>>>>>>> the only place I can think of. Can you please point me to any related
>>>>>>>> documentation?
>>>>>>>>
>>>>>>>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <
>>>>>>>> nikolaos.ballas@nexusgroup.com> wrote:
>>>>>>>>
>>>>>>>>> If you are not using any dns like service under
>>>>>>>>> /etc/mesos-master/ create two files called ip and hostname and put the ip
>>>>>>>>> of the eth interface.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Sent from my Samsung device
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -------- Original message --------
>>>>>>>>> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
>>>>>>>>> Date: 28/07/2015 20:18 (GMT+01:00)
>>>>>>>>> To: user@mesos.apache.org
>>>>>>>>> Subject: Problems connecting with Mesos Master
>>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>>>>>>>>>
>>>>>>>>> When I try to start my spark-shell, it gives me the following
>>>>>>>>> warning :
>>>>>>>>>
>>>>>>>>> **************************************************
>>>>>>>>>
>>>>>>>>> Scheduler driver bound to loopback interface! Cannot communicate
>>>>>>>>> with remote master(s). You might want to set 'LIBPROCESS_IP' environment
>>>>>>>>> variable to use a routable IP address.
>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>> Spark-shell works fine on the node where I run master, but if I
>>>>>>>>> start running on any of the other slave nodes it gives me the following
>>>>>>>>> error:
>>>>>>>>>
>>>>>>>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on
>>>>>>>>> fd=6: Transport endpoint is not connected [107]
>>>>>>>>>
>>>>>>>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on
>>>>>>>>> fd=6: Transport endpoint is not connected [107]
>>>>>>>>>
>>>>>>>>> I have the following configs:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - zookeeper configured to the mesos master
>>>>>>>>> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>>>>>>>>>
>>>>>>>>> I am not sure if I have to set the ip flag and where I have to
>>>>>>>>> set the --ip flag?
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Regards,
>>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Haripriya Ayyalasomayajula
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Haripriya Ayyalasomayajula
>>>>
>>>>
>>>
>>
>>
>> --
>> Regards,
>> Haripriya Ayyalasomayajula
>>
>>
>
--
Regards,
Haripriya Ayyalasomayajula
Re: Problems connecting with Mesos Master
Posted by Ondrej Smola <on...@gmail.com>.
Hi Haripiya,
When you run Spark on Mesos it needs to run
spark driver
mesos scheduler
and both need to be visible to outside world on public iface IP
you need to tell Spark and Mesos on which interface to bind - by default
they resolve node hostname to ip - this is loopback address in your case
Possible solutions - on slave node with public IP 192.168.56.50
1. Set
export LIBPROCESS_IP=192.168.56.50
export SPARK_LOCAL_IP=192.168.56.50
2. Ensure your hostname resolves to public iface IP - (for testing) edit
/etc/hosts to resolve your domain name to 192.168.56.50
3. Set correct hostname/ip in mesos configuration - see Nikolaos answer
2015-07-29 1:04 GMT+02:00 Haripriya Ayyalasomayajula <aharipriya92@gmail.com
>:
> It quits before it writes any logs, when I look at the directory for logs,
> it has empty files!
>
> On Tue, Jul 28, 2015 at 2:34 PM, Vinod Kone <vi...@gmail.com> wrote:
>
>> can you paste the logs?
>>
>> On Tue, Jul 28, 2015 at 2:31 PM, Haripriya Ayyalasomayajula <
>> aharipriya92@gmail.com> wrote:
>>
>>> Well, when I try doing that I get this error:
>>>
>>> Failed to initialize: Failed to bind on scheduler_ip_address Cannot
>>> assign requested address: Cannot assign requested address [99]
>>>
>>> When I do a
>>>
>>> ps -ef | grep mesos
>>>
>>> on both my master and slave nodes, it works fine. And, I am also able to
>>> ping both the nodes from each other - they are reachable to one another.
>>>
>>> On Tue, Jul 28, 2015 at 2:18 PM, Vinod Kone <vi...@gmail.com> wrote:
>>>
>>>> LIBPROCESS_IP is the IP address that you want the scheduler (driver) to
>>>> bind to. It has nothing to do with the ZooKeeper address.
>>>>
>>>> In other words, do
>>>>
>>>> export LIBPROCESS_IP=scheduler_ip_address
>>>>
>>>>
>>>> On Tue, Jul 28, 2015 at 2:00 PM, Haripriya Ayyalasomayajula <
>>>> aharipriya92@gmail.com> wrote:
>>>>
>>>>> I am trying to do this
>>>>>
>>>>> export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos
>>>>>
>>>>> ./bin/spark-shell
>>>>>
>>>>> It gives me this error and aborts
>>>>>
>>>>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>>>>>
>>>>> F0728 15:43:39.361445 13209 process.cpp:847] Parsing
>>>>> LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP
>>>>>
>>>>> On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <ti...@mesosphere.io> wrote:
>>>>>
>>>>>> spark-env.sh works as it will be called by spark-submit/spark-shell,
>>>>>> or you can just set it before you call spark-shell yourself.
>>>>>>
>>>>>> Tim
>>>>>>
>>>>>> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula <
>>>>>> aharipriya92@gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Where can I set the libprocess_ip env variable? spark_env.sh? Thats
>>>>>>> the only place I can think of. Can you please point me to any related
>>>>>>> documentation?
>>>>>>>
>>>>>>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <
>>>>>>> nikolaos.ballas@nexusgroup.com> wrote:
>>>>>>>
>>>>>>>> If you are not using any dns like service under
>>>>>>>> /etc/mesos-master/ create two files called ip and hostname and put the ip
>>>>>>>> of the eth interface.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Sent from my Samsung device
>>>>>>>>
>>>>>>>>
>>>>>>>> -------- Original message --------
>>>>>>>> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
>>>>>>>> Date: 28/07/2015 20:18 (GMT+01:00)
>>>>>>>> To: user@mesos.apache.org
>>>>>>>> Subject: Problems connecting with Mesos Master
>>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>>>>>>>>
>>>>>>>> When I try to start my spark-shell, it gives me the following
>>>>>>>> warning :
>>>>>>>>
>>>>>>>> **************************************************
>>>>>>>>
>>>>>>>> Scheduler driver bound to loopback interface! Cannot communicate
>>>>>>>> with remote master(s). You might want to set 'LIBPROCESS_IP' environment
>>>>>>>> variable to use a routable IP address.
>>>>>>>> ---
>>>>>>>>
>>>>>>>> Spark-shell works fine on the node where I run master, but if I
>>>>>>>> start running on any of the other slave nodes it gives me the following
>>>>>>>> error:
>>>>>>>>
>>>>>>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on
>>>>>>>> fd=6: Transport endpoint is not connected [107]
>>>>>>>>
>>>>>>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on
>>>>>>>> fd=6: Transport endpoint is not connected [107]
>>>>>>>>
>>>>>>>> I have the following configs:
>>>>>>>>
>>>>>>>>
>>>>>>>> - zookeeper configured to the mesos master
>>>>>>>> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>>>>>>>>
>>>>>>>> I am not sure if I have to set the ip flag and where I have to
>>>>>>>> set the --ip flag?
>>>>>>>>
>>>>>>>> --
>>>>>>>> Regards,
>>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Regards,
>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Haripriya Ayyalasomayajula
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Haripriya Ayyalasomayajula
>>>
>>>
>>
>
>
> --
> Regards,
> Haripriya Ayyalasomayajula
>
>
Re: Problems connecting with Mesos Master
Posted by Haripriya Ayyalasomayajula <ah...@gmail.com>.
It quits before it writes any logs, when I look at the directory for logs,
it has empty files!
On Tue, Jul 28, 2015 at 2:34 PM, Vinod Kone <vi...@gmail.com> wrote:
> can you paste the logs?
>
> On Tue, Jul 28, 2015 at 2:31 PM, Haripriya Ayyalasomayajula <
> aharipriya92@gmail.com> wrote:
>
>> Well, when I try doing that I get this error:
>>
>> Failed to initialize: Failed to bind on scheduler_ip_address Cannot
>> assign requested address: Cannot assign requested address [99]
>>
>> When I do a
>>
>> ps -ef | grep mesos
>>
>> on both my master and slave nodes, it works fine. And, I am also able to
>> ping both the nodes from each other - they are reachable to one another.
>>
>> On Tue, Jul 28, 2015 at 2:18 PM, Vinod Kone <vi...@gmail.com> wrote:
>>
>>> LIBPROCESS_IP is the IP address that you want the scheduler (driver) to
>>> bind to. It has nothing to do with the ZooKeeper address.
>>>
>>> In other words, do
>>>
>>> export LIBPROCESS_IP=scheduler_ip_address
>>>
>>>
>>> On Tue, Jul 28, 2015 at 2:00 PM, Haripriya Ayyalasomayajula <
>>> aharipriya92@gmail.com> wrote:
>>>
>>>> I am trying to do this
>>>>
>>>> export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos
>>>>
>>>> ./bin/spark-shell
>>>>
>>>> It gives me this error and aborts
>>>>
>>>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>>>>
>>>> F0728 15:43:39.361445 13209 process.cpp:847] Parsing
>>>> LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP
>>>>
>>>> On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <ti...@mesosphere.io> wrote:
>>>>
>>>>> spark-env.sh works as it will be called by spark-submit/spark-shell,
>>>>> or you can just set it before you call spark-shell yourself.
>>>>>
>>>>> Tim
>>>>>
>>>>> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula <
>>>>> aharipriya92@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Where can I set the libprocess_ip env variable? spark_env.sh? Thats
>>>>>> the only place I can think of. Can you please point me to any related
>>>>>> documentation?
>>>>>>
>>>>>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <
>>>>>> nikolaos.ballas@nexusgroup.com> wrote:
>>>>>>
>>>>>>> If you are not using any dns like service under
>>>>>>> /etc/mesos-master/ create two files called ip and hostname and put the ip
>>>>>>> of the eth interface.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Sent from my Samsung device
>>>>>>>
>>>>>>>
>>>>>>> -------- Original message --------
>>>>>>> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
>>>>>>> Date: 28/07/2015 20:18 (GMT+01:00)
>>>>>>> To: user@mesos.apache.org
>>>>>>> Subject: Problems connecting with Mesos Master
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>>>>>>>
>>>>>>> When I try to start my spark-shell, it gives me the following
>>>>>>> warning :
>>>>>>>
>>>>>>> **************************************************
>>>>>>>
>>>>>>> Scheduler driver bound to loopback interface! Cannot communicate
>>>>>>> with remote master(s). You might want to set 'LIBPROCESS_IP' environment
>>>>>>> variable to use a routable IP address.
>>>>>>> ---
>>>>>>>
>>>>>>> Spark-shell works fine on the node where I run master, but if I
>>>>>>> start running on any of the other slave nodes it gives me the following
>>>>>>> error:
>>>>>>>
>>>>>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on
>>>>>>> fd=6: Transport endpoint is not connected [107]
>>>>>>>
>>>>>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6:
>>>>>>> Transport endpoint is not connected [107]
>>>>>>>
>>>>>>> I have the following configs:
>>>>>>>
>>>>>>>
>>>>>>> - zookeeper configured to the mesos master
>>>>>>> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>>>>>>>
>>>>>>> I am not sure if I have to set the ip flag and where I have to set
>>>>>>> the --ip flag?
>>>>>>>
>>>>>>> --
>>>>>>> Regards,
>>>>>>> Haripriya Ayyalasomayajula
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Haripriya Ayyalasomayajula
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Haripriya Ayyalasomayajula
>>>>
>>>>
>>>
>>
>>
>> --
>> Regards,
>> Haripriya Ayyalasomayajula
>>
>>
>
--
Regards,
Haripriya Ayyalasomayajula
Re: Problems connecting with Mesos Master
Posted by Vinod Kone <vi...@gmail.com>.
can you paste the logs?
On Tue, Jul 28, 2015 at 2:31 PM, Haripriya Ayyalasomayajula <
aharipriya92@gmail.com> wrote:
> Well, when I try doing that I get this error:
>
> Failed to initialize: Failed to bind on scheduler_ip_address Cannot assign
> requested address: Cannot assign requested address [99]
>
> When I do a
>
> ps -ef | grep mesos
>
> on both my master and slave nodes, it works fine. And, I am also able to
> ping both the nodes from each other - they are reachable to one another.
>
> On Tue, Jul 28, 2015 at 2:18 PM, Vinod Kone <vi...@gmail.com> wrote:
>
>> LIBPROCESS_IP is the IP address that you want the scheduler (driver) to
>> bind to. It has nothing to do with the ZooKeeper address.
>>
>> In other words, do
>>
>> export LIBPROCESS_IP=scheduler_ip_address
>>
>>
>> On Tue, Jul 28, 2015 at 2:00 PM, Haripriya Ayyalasomayajula <
>> aharipriya92@gmail.com> wrote:
>>
>>> I am trying to do this
>>>
>>> export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos
>>>
>>> ./bin/spark-shell
>>>
>>> It gives me this error and aborts
>>>
>>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>>>
>>> F0728 15:43:39.361445 13209 process.cpp:847] Parsing
>>> LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP
>>>
>>> On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <ti...@mesosphere.io> wrote:
>>>
>>>> spark-env.sh works as it will be called by spark-submit/spark-shell, or
>>>> you can just set it before you call spark-shell yourself.
>>>>
>>>> Tim
>>>>
>>>> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula <
>>>> aharipriya92@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Where can I set the libprocess_ip env variable? spark_env.sh? Thats
>>>>> the only place I can think of. Can you please point me to any related
>>>>> documentation?
>>>>>
>>>>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <
>>>>> nikolaos.ballas@nexusgroup.com> wrote:
>>>>>
>>>>>> If you are not using any dns like service under /etc/mesos-master/
>>>>>> create two files called ip and hostname and put the ip of the eth
>>>>>> interface.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Sent from my Samsung device
>>>>>>
>>>>>>
>>>>>> -------- Original message --------
>>>>>> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
>>>>>> Date: 28/07/2015 20:18 (GMT+01:00)
>>>>>> To: user@mesos.apache.org
>>>>>> Subject: Problems connecting with Mesos Master
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>>>>>>
>>>>>> When I try to start my spark-shell, it gives me the following
>>>>>> warning :
>>>>>>
>>>>>> **************************************************
>>>>>>
>>>>>> Scheduler driver bound to loopback interface! Cannot communicate with
>>>>>> remote master(s). You might want to set 'LIBPROCESS_IP' environment
>>>>>> variable to use a routable IP address.
>>>>>> ---
>>>>>>
>>>>>> Spark-shell works fine on the node where I run master, but if I
>>>>>> start running on any of the other slave nodes it gives me the following
>>>>>> error:
>>>>>>
>>>>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on
>>>>>> fd=6: Transport endpoint is not connected [107]
>>>>>>
>>>>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6:
>>>>>> Transport endpoint is not connected [107]
>>>>>>
>>>>>> I have the following configs:
>>>>>>
>>>>>>
>>>>>> - zookeeper configured to the mesos master
>>>>>> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>>>>>>
>>>>>> I am not sure if I have to set the ip flag and where I have to set
>>>>>> the --ip flag?
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Haripriya Ayyalasomayajula
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Haripriya Ayyalasomayajula
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Haripriya Ayyalasomayajula
>>>
>>>
>>
>
>
> --
> Regards,
> Haripriya Ayyalasomayajula
>
>
Re: Problems connecting with Mesos Master
Posted by Haripriya Ayyalasomayajula <ah...@gmail.com>.
Well, when I try doing that I get this error:
Failed to initialize: Failed to bind on scheduler_ip_address Cannot assign
requested address: Cannot assign requested address [99]
When I do a
ps -ef | grep mesos
on both my master and slave nodes, it works fine. And, I am also able to
ping both the nodes from each other - they are reachable to one another.
On Tue, Jul 28, 2015 at 2:18 PM, Vinod Kone <vi...@gmail.com> wrote:
> LIBPROCESS_IP is the IP address that you want the scheduler (driver) to
> bind to. It has nothing to do with the ZooKeeper address.
>
> In other words, do
>
> export LIBPROCESS_IP=scheduler_ip_address
>
>
> On Tue, Jul 28, 2015 at 2:00 PM, Haripriya Ayyalasomayajula <
> aharipriya92@gmail.com> wrote:
>
>> I am trying to do this
>>
>> export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos
>>
>> ./bin/spark-shell
>>
>> It gives me this error and aborts
>>
>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>>
>> F0728 15:43:39.361445 13209 process.cpp:847] Parsing
>> LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP
>>
>> On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <ti...@mesosphere.io> wrote:
>>
>>> spark-env.sh works as it will be called by spark-submit/spark-shell, or
>>> you can just set it before you call spark-shell yourself.
>>>
>>> Tim
>>>
>>> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula <
>>> aharipriya92@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Where can I set the libprocess_ip env variable? spark_env.sh? Thats the
>>>> only place I can think of. Can you please point me to any related
>>>> documentation?
>>>>
>>>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <
>>>> nikolaos.ballas@nexusgroup.com> wrote:
>>>>
>>>>> If you are not using any dns like service under /etc/mesos-master/
>>>>> create two files called ip and hostname and put the ip of the eth
>>>>> interface.
>>>>>
>>>>>
>>>>>
>>>>> Sent from my Samsung device
>>>>>
>>>>>
>>>>> -------- Original message --------
>>>>> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
>>>>> Date: 28/07/2015 20:18 (GMT+01:00)
>>>>> To: user@mesos.apache.org
>>>>> Subject: Problems connecting with Mesos Master
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>>>>>
>>>>> When I try to start my spark-shell, it gives me the following
>>>>> warning :
>>>>>
>>>>> **************************************************
>>>>>
>>>>> Scheduler driver bound to loopback interface! Cannot communicate with
>>>>> remote master(s). You might want to set 'LIBPROCESS_IP' environment
>>>>> variable to use a routable IP address.
>>>>> ---
>>>>>
>>>>> Spark-shell works fine on the node where I run master, but if I
>>>>> start running on any of the other slave nodes it gives me the following
>>>>> error:
>>>>>
>>>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on fd=6:
>>>>> Transport endpoint is not connected [107]
>>>>>
>>>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6:
>>>>> Transport endpoint is not connected [107]
>>>>>
>>>>> I have the following configs:
>>>>>
>>>>>
>>>>> - zookeeper configured to the mesos master
>>>>> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>>>>>
>>>>> I am not sure if I have to set the ip flag and where I have to set
>>>>> the --ip flag?
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Haripriya Ayyalasomayajula
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Haripriya Ayyalasomayajula
>>>>
>>>>
>>>
>>
>>
>> --
>> Regards,
>> Haripriya Ayyalasomayajula
>>
>>
>
--
Regards,
Haripriya Ayyalasomayajula
Re: Problems connecting with Mesos Master
Posted by Vinod Kone <vi...@gmail.com>.
LIBPROCESS_IP is the IP address that you want the scheduler (driver) to
bind to. It has nothing to do with the ZooKeeper address.
In other words, do
export LIBPROCESS_IP=scheduler_ip_address
On Tue, Jul 28, 2015 at 2:00 PM, Haripriya Ayyalasomayajula <
aharipriya92@gmail.com> wrote:
> I am trying to do this
>
> export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos
>
> ./bin/spark-shell
>
> It gives me this error and aborts
>
> WARNING: Logging before InitGoogleLogging() is written to STDERR
>
> F0728 15:43:39.361445 13209 process.cpp:847] Parsing
> LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP
>
> On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <ti...@mesosphere.io> wrote:
>
>> spark-env.sh works as it will be called by spark-submit/spark-shell, or
>> you can just set it before you call spark-shell yourself.
>>
>> Tim
>>
>> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula <
>> aharipriya92@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Where can I set the libprocess_ip env variable? spark_env.sh? Thats the
>>> only place I can think of. Can you please point me to any related
>>> documentation?
>>>
>>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <
>>> nikolaos.ballas@nexusgroup.com> wrote:
>>>
>>>> If you are not using any dns like service under /etc/mesos-master/
>>>> create two files called ip and hostname and put the ip of the eth
>>>> interface.
>>>>
>>>>
>>>>
>>>> Sent from my Samsung device
>>>>
>>>>
>>>> -------- Original message --------
>>>> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
>>>> Date: 28/07/2015 20:18 (GMT+01:00)
>>>> To: user@mesos.apache.org
>>>> Subject: Problems connecting with Mesos Master
>>>>
>>>> Hi all,
>>>>
>>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>>>>
>>>> When I try to start my spark-shell, it gives me the following warning
>>>> :
>>>>
>>>> **************************************************
>>>>
>>>> Scheduler driver bound to loopback interface! Cannot communicate with
>>>> remote master(s). You might want to set 'LIBPROCESS_IP' environment
>>>> variable to use a routable IP address.
>>>> ---
>>>>
>>>> Spark-shell works fine on the node where I run master, but if I start
>>>> running on any of the other slave nodes it gives me the following error:
>>>>
>>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on fd=6:
>>>> Transport endpoint is not connected [107]
>>>>
>>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6:
>>>> Transport endpoint is not connected [107]
>>>>
>>>> I have the following configs:
>>>>
>>>>
>>>> - zookeeper configured to the mesos master
>>>> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>>>>
>>>> I am not sure if I have to set the ip flag and where I have to set
>>>> the --ip flag?
>>>>
>>>> --
>>>> Regards,
>>>> Haripriya Ayyalasomayajula
>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Haripriya Ayyalasomayajula
>>>
>>>
>>
>
>
> --
> Regards,
> Haripriya Ayyalasomayajula
>
>
Re: Problems connecting with Mesos Master
Posted by Haripriya Ayyalasomayajula <ah...@gmail.com>.
I am trying to do this
export LIBPROCESS_IP=zk://my_ipaddress:2181/mesos
./bin/spark-shell
It gives me this error and aborts
WARNING: Logging before InitGoogleLogging() is written to STDERR
F0728 15:43:39.361445 13209 process.cpp:847] Parsing
LIBPROCESS_IP=zk://my_ipaddress:2181/ failed: Failed to parse the IP
On Tue, Jul 28, 2015 at 1:51 PM, Tim Chen <ti...@mesosphere.io> wrote:
> spark-env.sh works as it will be called by spark-submit/spark-shell, or
> you can just set it before you call spark-shell yourself.
>
> Tim
>
> On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula <
> aharipriya92@gmail.com> wrote:
>
>> Hi,
>>
>> Where can I set the libprocess_ip env variable? spark_env.sh? Thats the
>> only place I can think of. Can you please point me to any related
>> documentation?
>>
>> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <
>> nikolaos.ballas@nexusgroup.com> wrote:
>>
>>> If you are not using any dns like service under /etc/mesos-master/
>>> create two files called ip and hostname and put the ip of the eth
>>> interface.
>>>
>>>
>>>
>>> Sent from my Samsung device
>>>
>>>
>>> -------- Original message --------
>>> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
>>> Date: 28/07/2015 20:18 (GMT+01:00)
>>> To: user@mesos.apache.org
>>> Subject: Problems connecting with Mesos Master
>>>
>>> Hi all,
>>>
>>> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>>>
>>> When I try to start my spark-shell, it gives me the following warning :
>>>
>>> **************************************************
>>>
>>> Scheduler driver bound to loopback interface! Cannot communicate with
>>> remote master(s). You might want to set 'LIBPROCESS_IP' environment
>>> variable to use a routable IP address.
>>> ---
>>>
>>> Spark-shell works fine on the node where I run master, but if I start
>>> running on any of the other slave nodes it gives me the following error:
>>>
>>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on fd=6:
>>> Transport endpoint is not connected [107]
>>>
>>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6:
>>> Transport endpoint is not connected [107]
>>>
>>> I have the following configs:
>>>
>>>
>>> - zookeeper configured to the mesos master
>>> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>>>
>>> I am not sure if I have to set the ip flag and where I have to set the
>>> --ip flag?
>>>
>>> --
>>> Regards,
>>> Haripriya Ayyalasomayajula
>>>
>>>
>>
>>
>> --
>> Regards,
>> Haripriya Ayyalasomayajula
>>
>>
>
--
Regards,
Haripriya Ayyalasomayajula
Re: Problems connecting with Mesos Master
Posted by Tim Chen <ti...@mesosphere.io>.
spark-env.sh works as it will be called by spark-submit/spark-shell, or you
can just set it before you call spark-shell yourself.
Tim
On Tue, Jul 28, 2015 at 1:43 PM, Haripriya Ayyalasomayajula <
aharipriya92@gmail.com> wrote:
> Hi,
>
> Where can I set the libprocess_ip env variable? spark_env.sh? Thats the
> only place I can think of. Can you please point me to any related
> documentation?
>
> On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <
> nikolaos.ballas@nexusgroup.com> wrote:
>
>> If you are not using any dns like service under /etc/mesos-master/
>> create two files called ip and hostname and put the ip of the eth
>> interface.
>>
>>
>>
>> Sent from my Samsung device
>>
>>
>> -------- Original message --------
>> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
>> Date: 28/07/2015 20:18 (GMT+01:00)
>> To: user@mesos.apache.org
>> Subject: Problems connecting with Mesos Master
>>
>> Hi all,
>>
>> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>>
>> When I try to start my spark-shell, it gives me the following warning :
>>
>> **************************************************
>>
>> Scheduler driver bound to loopback interface! Cannot communicate with
>> remote master(s). You might want to set 'LIBPROCESS_IP' environment
>> variable to use a routable IP address.
>> ---
>>
>> Spark-shell works fine on the node where I run master, but if I start
>> running on any of the other slave nodes it gives me the following error:
>>
>> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on fd=6:
>> Transport endpoint is not connected [107]
>>
>> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6:
>> Transport endpoint is not connected [107]
>>
>> I have the following configs:
>>
>>
>> - zookeeper configured to the mesos master
>> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>>
>> I am not sure if I have to set the ip flag and where I have to set the
>> --ip flag?
>>
>> --
>> Regards,
>> Haripriya Ayyalasomayajula
>>
>>
>
>
> --
> Regards,
> Haripriya Ayyalasomayajula
>
>
Re: Problems connecting with Mesos Master
Posted by Haripriya Ayyalasomayajula <ah...@gmail.com>.
Hi,
Where can I set the libprocess_ip env variable? spark_env.sh? Thats the
only place I can think of. Can you please point me to any related
documentation?
On Tue, Jul 28, 2015 at 12:46 PM, Nikolaos Ballas neXus <
nikolaos.ballas@nexusgroup.com> wrote:
> If you are not using any dns like service under /etc/mesos-master/
> create two files called ip and hostname and put the ip of the eth
> interface.
>
>
>
> Sent from my Samsung device
>
>
> -------- Original message --------
> From: Haripriya Ayyalasomayajula <ah...@gmail.com>
> Date: 28/07/2015 20:18 (GMT+01:00)
> To: user@mesos.apache.org
> Subject: Problems connecting with Mesos Master
>
> Hi all,
>
> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>
> When I try to start my spark-shell, it gives me the following warning :
>
> **************************************************
>
> Scheduler driver bound to loopback interface! Cannot communicate with
> remote master(s). You might want to set 'LIBPROCESS_IP' environment
> variable to use a routable IP address.
> ---
>
> Spark-shell works fine on the node where I run master, but if I start
> running on any of the other slave nodes it gives me the following error:
>
> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on fd=6:
> Transport endpoint is not connected [107]
>
> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6:
> Transport endpoint is not connected [107]
>
> I have the following configs:
>
>
> - zookeeper configured to the mesos master
> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>
> I am not sure if I have to set the ip flag and where I have to set the
> --ip flag?
>
> --
> Regards,
> Haripriya Ayyalasomayajula
>
>
--
Regards,
Haripriya Ayyalasomayajula
RE: Problems connecting with Mesos Master
Posted by Nikolaos Ballas neXus <ni...@nexusgroup.com>.
If you are not using any dns like service under /etc/mesos-master/ create two files called ip and hostname and put the ip of the eth interface.
Sent from my Samsung device
-------- Original message --------
From: Haripriya Ayyalasomayajula <ah...@gmail.com>
Date: 28/07/2015 20:18 (GMT+01:00)
To: user@mesos.apache.org
Subject: Problems connecting with Mesos Master
Hi all,
I am trying to use Spark 1.4.1 with Mesos 0.23.0.
When I try to start my spark-shell, it gives me the following warning :
**************************************************
Scheduler driver bound to loopback interface! Cannot communicate with remote master(s). You might want to set 'LIBPROCESS_IP' environment variable to use a routable IP address.
---
Spark-shell works fine on the node where I run master, but if I start running on any of the other slave nodes it gives me the following error:
E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on fd=6: Transport endpoint is not connected [107]
E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6: Transport endpoint is not connected [107]
I have the following configs:
* zookeeper configured to the mesos master
* /etc/mesos/zk on all nodes pointing to mesos master ip.
I am not sure if I have to set the ip flag and where I have to set the --ip flag?
--
Regards,
Haripriya Ayyalasomayajula
Re: Problems connecting with Mesos Master
Posted by Vinod Kone <vi...@gmail.com>.
did you set LIBPROCESS_IP env variable as the warning suggested?
On Tue, Jul 28, 2015 at 11:16 AM, Haripriya Ayyalasomayajula <
aharipriya92@gmail.com> wrote:
> Hi all,
>
> I am trying to use Spark 1.4.1 with Mesos 0.23.0.
>
> When I try to start my spark-shell, it gives me the following warning :
>
> **************************************************
>
> Scheduler driver bound to loopback interface! Cannot communicate with
> remote master(s). You might want to set 'LIBPROCESS_IP' environment
> variable to use a routable IP address.
> ---
>
> Spark-shell works fine on the node where I run master, but if I start
> running on any of the other slave nodes it gives me the following error:
>
> E0728 11:22:53.176515 10503 socket.hpp:107] Shutdown failed on fd=6:
> Transport endpoint is not connected [107]
>
> E0728 11:22:53.210146 10503 socket.hpp:107] Shutdown failed on fd=6:
> Transport endpoint is not connected [107]
>
> I have the following configs:
>
>
> - zookeeper configured to the mesos master
> - /etc/mesos/zk on all nodes pointing to mesos master ip.
>
> I am not sure if I have to set the ip flag and where I have to set the
> --ip flag?
>
> --
> Regards,
> Haripriya Ayyalasomayajula
>
>