You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Khalid Goudeaux <kh...@imc-chicago.com> on 2013/11/08 22:28:37 UTC
mesos in a docker container
Is it possible to control the ip that the mesos-master publishes to
zookeeper for election/discovery? When running inside a docker container
the hostname and ip that the master sees may not be the host/ip that it's
externally accessible from.
Also, inside a container the master would bind to and publish 127.0.0.1 by
default instead of the container ip as expected. Switching to 0.0.0.0 had
no effect. Only by explicitly specifying the container ip would it bind and
publish a usable ip.
Thanks,
Khalid
Re: mesos in a docker container
Posted by Khalid Goudeaux <kh...@imc-chicago.com>.
Filed: https://issues.apache.org/jira/browse/MESOS-809
I am attempting to run everything inside containers. The idea is to have
one mechanism for installing/running software regardless of type, linux
distro or dependencies.
The CoreOS project is an interesting example of this - the OS has no
packaging system and the root filesystem is readonly. Applications can only
run via docker containers.
On Fri, Nov 8, 2013 at 6:06 PM, Vinod Kone <vi...@gmail.com> wrote:
> Ok. Mind filing a ticket?
>
> Also, I'm curious about your setup. Are you running each mesos component
> (master, slave, framework) in a docker container? If yes, how come?
>
>
> On Fri, Nov 8, 2013 at 4:00 PM, Khalid Goudeaux <
> khalid.goudeaux@imc-chicago.com> wrote:
>
>> Yes, it uses NAT to go from the public ip/port to a private ip/port.
>>
>>
>> On Fri, Nov 8, 2013 at 5:36 PM, Vinod Kone <vi...@gmail.com> wrote:
>>
>>> I see. So does docker handle the routing if the master binds to a
>>> container private port but frameworks/slaves try to connect with its public
>>> ip?
>>>
>>>
>>> On Fri, Nov 8, 2013 at 2:25 PM, Khalid Goudeaux <
>>> khalid.goudeaux@imc-chicago.com> wrote:
>>>
>>>> From inside a container, the master will see an ip and publish this to
>>>> zookeeper. That ip, however, is only accessible from within the machine
>>>> with the container. No other machine on the network can access it.
>>>>
>>>> I can use the --ip option, but from within the container I can't bind
>>>> to the ip of the host system (nor do I want to as the container isolation
>>>> is useful). Essentially the master is registering itself in Zookeeper but
>>>> doesn't authoritatively know where it is.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Nov 8, 2013 at 3:36 PM, Vinod Kone <vi...@gmail.com>wrote:
>>>>
>>>>> Not sure I understand your question. From the last sentence it
>>>>> looks like you already know about the "--ip" flag that could be passed to
>>>>> the master. Are you looking for something else?
>>>>>
>>>>>
>>>>> On Fri, Nov 8, 2013 at 1:28 PM, Khalid Goudeaux <
>>>>> khalid.goudeaux@imc-chicago.com> wrote:
>>>>>
>>>>>> Is it possible to control the ip that the mesos-master publishes to
>>>>>> zookeeper for election/discovery? When running inside a docker container
>>>>>> the hostname and ip that the master sees may not be the host/ip that it's
>>>>>> externally accessible from.
>>>>>>
>>>>>> Also, inside a container the master would bind to and publish
>>>>>> 127.0.0.1 by default instead of the container ip as expected. Switching to
>>>>>> 0.0.0.0 had no effect. Only by explicitly specifying the container ip would
>>>>>> it bind and publish a usable ip.
>>>>>>
>>>>>> Thanks,
>>>>>> Khalid
>>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> The information in this e-mail is intended only for the person or
>>>>> entity to which it is addressed.
>>>>>
>>>>> It may contain confidential and /or privileged material. If someone
>>>>> other than the intended recipient should receive this e-mail, he / she
>>>>> shall not be entitled to read, disseminate, disclose or duplicate it.
>>>>>
>>>>> If you receive this e-mail unintentionally, please inform us
>>>>> immediately by "reply" and then delete it from your system. Although this
>>>>> information has been compiled with great care, neither IMC Financial
>>>>> Markets & Asset Management nor any of its related entities shall accept any
>>>>> responsibility for any errors, omissions or other inaccuracies in this
>>>>> information or for the consequences thereof, nor shall it be bound in any
>>>>> way by the contents of this e-mail or its attachments. In the event of
>>>>> incomplete or incorrect transmission, please return the e-mail to the
>>>>> sender and permanently delete this message and any attachments.
>>>>>
>>>>> Messages and attachments are scanned for all known viruses. Always
>>>>> scan attachments before opening them.
>>>>>
>>>>
>>>>
>>>
>>> ------------------------------
>>>
>>> The information in this e-mail is intended only for the person or entity
>>> to which it is addressed.
>>>
>>> It may contain confidential and /or privileged material. If someone
>>> other than the intended recipient should receive this e-mail, he / she
>>> shall not be entitled to read, disseminate, disclose or duplicate it.
>>>
>>> If you receive this e-mail unintentionally, please inform us immediately
>>> by "reply" and then delete it from your system. Although this information
>>> has been compiled with great care, neither IMC Financial Markets & Asset
>>> Management nor any of its related entities shall accept any responsibility
>>> for any errors, omissions or other inaccuracies in this information or for
>>> the consequences thereof, nor shall it be bound in any way by the contents
>>> of this e-mail or its attachments. In the event of incomplete or incorrect
>>> transmission, please return the e-mail to the sender and permanently delete
>>> this message and any attachments.
>>>
>>> Messages and attachments are scanned for all known viruses. Always scan
>>> attachments before opening them.
>>>
>>
>>
>
> ------------------------------
>
> The information in this e-mail is intended only for the person or entity
> to which it is addressed.
>
> It may contain confidential and /or privileged material. If someone other
> than the intended recipient should receive this e-mail, he / she shall not
> be entitled to read, disseminate, disclose or duplicate it.
>
> If you receive this e-mail unintentionally, please inform us immediately
> by "reply" and then delete it from your system. Although this information
> has been compiled with great care, neither IMC Financial Markets & Asset
> Management nor any of its related entities shall accept any responsibility
> for any errors, omissions or other inaccuracies in this information or for
> the consequences thereof, nor shall it be bound in any way by the contents
> of this e-mail or its attachments. In the event of incomplete or incorrect
> transmission, please return the e-mail to the sender and permanently delete
> this message and any attachments.
>
> Messages and attachments are scanned for all known viruses. Always scan
> attachments before opening them.
>
Re: mesos in a docker container
Posted by Vinod Kone <vi...@gmail.com>.
Ok. Mind filing a ticket?
Also, I'm curious about your setup. Are you running each mesos component
(master, slave, framework) in a docker container? If yes, how come?
On Fri, Nov 8, 2013 at 4:00 PM, Khalid Goudeaux <
khalid.goudeaux@imc-chicago.com> wrote:
> Yes, it uses NAT to go from the public ip/port to a private ip/port.
>
>
> On Fri, Nov 8, 2013 at 5:36 PM, Vinod Kone <vi...@gmail.com> wrote:
>
>> I see. So does docker handle the routing if the master binds to a
>> container private port but frameworks/slaves try to connect with its public
>> ip?
>>
>>
>> On Fri, Nov 8, 2013 at 2:25 PM, Khalid Goudeaux <
>> khalid.goudeaux@imc-chicago.com> wrote:
>>
>>> From inside a container, the master will see an ip and publish this to
>>> zookeeper. That ip, however, is only accessible from within the machine
>>> with the container. No other machine on the network can access it.
>>>
>>> I can use the --ip option, but from within the container I can't bind
>>> to the ip of the host system (nor do I want to as the container isolation
>>> is useful). Essentially the master is registering itself in Zookeeper but
>>> doesn't authoritatively know where it is.
>>>
>>>
>>>
>>>
>>> On Fri, Nov 8, 2013 at 3:36 PM, Vinod Kone <vi...@gmail.com> wrote:
>>>
>>>> Not sure I understand your question. From the last sentence it looks
>>>> like you already know about the "--ip" flag that could be passed to the
>>>> master. Are you looking for something else?
>>>>
>>>>
>>>> On Fri, Nov 8, 2013 at 1:28 PM, Khalid Goudeaux <
>>>> khalid.goudeaux@imc-chicago.com> wrote:
>>>>
>>>>> Is it possible to control the ip that the mesos-master publishes to
>>>>> zookeeper for election/discovery? When running inside a docker container
>>>>> the hostname and ip that the master sees may not be the host/ip that it's
>>>>> externally accessible from.
>>>>>
>>>>> Also, inside a container the master would bind to and publish
>>>>> 127.0.0.1 by default instead of the container ip as expected. Switching to
>>>>> 0.0.0.0 had no effect. Only by explicitly specifying the container ip would
>>>>> it bind and publish a usable ip.
>>>>>
>>>>> Thanks,
>>>>> Khalid
>>>>>
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> The information in this e-mail is intended only for the person or
>>>> entity to which it is addressed.
>>>>
>>>> It may contain confidential and /or privileged material. If someone
>>>> other than the intended recipient should receive this e-mail, he / she
>>>> shall not be entitled to read, disseminate, disclose or duplicate it.
>>>>
>>>> If you receive this e-mail unintentionally, please inform us
>>>> immediately by "reply" and then delete it from your system. Although this
>>>> information has been compiled with great care, neither IMC Financial
>>>> Markets & Asset Management nor any of its related entities shall accept any
>>>> responsibility for any errors, omissions or other inaccuracies in this
>>>> information or for the consequences thereof, nor shall it be bound in any
>>>> way by the contents of this e-mail or its attachments. In the event of
>>>> incomplete or incorrect transmission, please return the e-mail to the
>>>> sender and permanently delete this message and any attachments.
>>>>
>>>> Messages and attachments are scanned for all known viruses. Always scan
>>>> attachments before opening them.
>>>>
>>>
>>>
>>
>> ------------------------------
>>
>> The information in this e-mail is intended only for the person or entity
>> to which it is addressed.
>>
>> It may contain confidential and /or privileged material. If someone other
>> than the intended recipient should receive this e-mail, he / she shall not
>> be entitled to read, disseminate, disclose or duplicate it.
>>
>> If you receive this e-mail unintentionally, please inform us immediately
>> by "reply" and then delete it from your system. Although this information
>> has been compiled with great care, neither IMC Financial Markets & Asset
>> Management nor any of its related entities shall accept any responsibility
>> for any errors, omissions or other inaccuracies in this information or for
>> the consequences thereof, nor shall it be bound in any way by the contents
>> of this e-mail or its attachments. In the event of incomplete or incorrect
>> transmission, please return the e-mail to the sender and permanently delete
>> this message and any attachments.
>>
>> Messages and attachments are scanned for all known viruses. Always scan
>> attachments before opening them.
>>
>
>
Re: mesos in a docker container
Posted by Khalid Goudeaux <kh...@imc-chicago.com>.
Yes, it uses NAT to go from the public ip/port to a private ip/port.
On Fri, Nov 8, 2013 at 5:36 PM, Vinod Kone <vi...@gmail.com> wrote:
> I see. So does docker handle the routing if the master binds to a
> container private port but frameworks/slaves try to connect with its public
> ip?
>
>
> On Fri, Nov 8, 2013 at 2:25 PM, Khalid Goudeaux <
> khalid.goudeaux@imc-chicago.com> wrote:
>
>> From inside a container, the master will see an ip and publish this to
>> zookeeper. That ip, however, is only accessible from within the machine
>> with the container. No other machine on the network can access it.
>>
>> I can use the --ip option, but from within the container I can't bind
>> to the ip of the host system (nor do I want to as the container isolation
>> is useful). Essentially the master is registering itself in Zookeeper but
>> doesn't authoritatively know where it is.
>>
>>
>>
>>
>> On Fri, Nov 8, 2013 at 3:36 PM, Vinod Kone <vi...@gmail.com> wrote:
>>
>>> Not sure I understand your question. From the last sentence it looks
>>> like you already know about the "--ip" flag that could be passed to the
>>> master. Are you looking for something else?
>>>
>>>
>>> On Fri, Nov 8, 2013 at 1:28 PM, Khalid Goudeaux <
>>> khalid.goudeaux@imc-chicago.com> wrote:
>>>
>>>> Is it possible to control the ip that the mesos-master publishes to
>>>> zookeeper for election/discovery? When running inside a docker container
>>>> the hostname and ip that the master sees may not be the host/ip that it's
>>>> externally accessible from.
>>>>
>>>> Also, inside a container the master would bind to and publish
>>>> 127.0.0.1 by default instead of the container ip as expected. Switching to
>>>> 0.0.0.0 had no effect. Only by explicitly specifying the container ip would
>>>> it bind and publish a usable ip.
>>>>
>>>> Thanks,
>>>> Khalid
>>>>
>>>
>>>
>>> ------------------------------
>>>
>>> The information in this e-mail is intended only for the person or entity
>>> to which it is addressed.
>>>
>>> It may contain confidential and /or privileged material. If someone
>>> other than the intended recipient should receive this e-mail, he / she
>>> shall not be entitled to read, disseminate, disclose or duplicate it.
>>>
>>> If you receive this e-mail unintentionally, please inform us immediately
>>> by "reply" and then delete it from your system. Although this information
>>> has been compiled with great care, neither IMC Financial Markets & Asset
>>> Management nor any of its related entities shall accept any responsibility
>>> for any errors, omissions or other inaccuracies in this information or for
>>> the consequences thereof, nor shall it be bound in any way by the contents
>>> of this e-mail or its attachments. In the event of incomplete or incorrect
>>> transmission, please return the e-mail to the sender and permanently delete
>>> this message and any attachments.
>>>
>>> Messages and attachments are scanned for all known viruses. Always scan
>>> attachments before opening them.
>>>
>>
>>
>
> ------------------------------
>
> The information in this e-mail is intended only for the person or entity
> to which it is addressed.
>
> It may contain confidential and /or privileged material. If someone other
> than the intended recipient should receive this e-mail, he / she shall not
> be entitled to read, disseminate, disclose or duplicate it.
>
> If you receive this e-mail unintentionally, please inform us immediately
> by "reply" and then delete it from your system. Although this information
> has been compiled with great care, neither IMC Financial Markets & Asset
> Management nor any of its related entities shall accept any responsibility
> for any errors, omissions or other inaccuracies in this information or for
> the consequences thereof, nor shall it be bound in any way by the contents
> of this e-mail or its attachments. In the event of incomplete or incorrect
> transmission, please return the e-mail to the sender and permanently delete
> this message and any attachments.
>
> Messages and attachments are scanned for all known viruses. Always scan
> attachments before opening them.
>
Re: mesos in a docker container
Posted by Vinod Kone <vi...@gmail.com>.
I see. So does docker handle the routing if the master binds to a container
private port but frameworks/slaves try to connect with its public ip?
On Fri, Nov 8, 2013 at 2:25 PM, Khalid Goudeaux <
khalid.goudeaux@imc-chicago.com> wrote:
> From inside a container, the master will see an ip and publish this to
> zookeeper. That ip, however, is only accessible from within the machine
> with the container. No other machine on the network can access it.
>
> I can use the --ip option, but from within the container I can't bind to
> the ip of the host system (nor do I want to as the container isolation is
> useful). Essentially the master is registering itself in Zookeeper but
> doesn't authoritatively know where it is.
>
>
>
>
> On Fri, Nov 8, 2013 at 3:36 PM, Vinod Kone <vi...@gmail.com> wrote:
>
>> Not sure I understand your question. From the last sentence it looks
>> like you already know about the "--ip" flag that could be passed to the
>> master. Are you looking for something else?
>>
>>
>> On Fri, Nov 8, 2013 at 1:28 PM, Khalid Goudeaux <
>> khalid.goudeaux@imc-chicago.com> wrote:
>>
>>> Is it possible to control the ip that the mesos-master publishes to
>>> zookeeper for election/discovery? When running inside a docker container
>>> the hostname and ip that the master sees may not be the host/ip that it's
>>> externally accessible from.
>>>
>>> Also, inside a container the master would bind to and publish
>>> 127.0.0.1 by default instead of the container ip as expected. Switching to
>>> 0.0.0.0 had no effect. Only by explicitly specifying the container ip would
>>> it bind and publish a usable ip.
>>>
>>> Thanks,
>>> Khalid
>>>
>>
>>
>> ------------------------------
>>
>> The information in this e-mail is intended only for the person or entity
>> to which it is addressed.
>>
>> It may contain confidential and /or privileged material. If someone other
>> than the intended recipient should receive this e-mail, he / she shall not
>> be entitled to read, disseminate, disclose or duplicate it.
>>
>> If you receive this e-mail unintentionally, please inform us immediately
>> by "reply" and then delete it from your system. Although this information
>> has been compiled with great care, neither IMC Financial Markets & Asset
>> Management nor any of its related entities shall accept any responsibility
>> for any errors, omissions or other inaccuracies in this information or for
>> the consequences thereof, nor shall it be bound in any way by the contents
>> of this e-mail or its attachments. In the event of incomplete or incorrect
>> transmission, please return the e-mail to the sender and permanently delete
>> this message and any attachments.
>>
>> Messages and attachments are scanned for all known viruses. Always scan
>> attachments before opening them.
>>
>
>
Re: mesos in a docker container
Posted by Khalid Goudeaux <kh...@imc-chicago.com>.
>From inside a container, the master will see an ip and publish this to
zookeeper. That ip, however, is only accessible from within the machine
with the container. No other machine on the network can access it.
I can use the --ip option, but from within the container I can't bind to
the ip of the host system (nor do I want to as the container isolation is
useful). Essentially the master is registering itself in Zookeeper but
doesn't authoritatively know where it is.
On Fri, Nov 8, 2013 at 3:36 PM, Vinod Kone <vi...@gmail.com> wrote:
> Not sure I understand your question. From the last sentence it looks
> like you already know about the "--ip" flag that could be passed to the
> master. Are you looking for something else?
>
>
> On Fri, Nov 8, 2013 at 1:28 PM, Khalid Goudeaux <
> khalid.goudeaux@imc-chicago.com> wrote:
>
>> Is it possible to control the ip that the mesos-master publishes to
>> zookeeper for election/discovery? When running inside a docker container
>> the hostname and ip that the master sees may not be the host/ip that it's
>> externally accessible from.
>>
>> Also, inside a container the master would bind to and publish 127.0.0.1
>> by default instead of the container ip as expected. Switching to 0.0.0.0
>> had no effect. Only by explicitly specifying the container ip would it bind
>> and publish a usable ip.
>>
>> Thanks,
>> Khalid
>>
>
>
> ------------------------------
>
> The information in this e-mail is intended only for the person or entity
> to which it is addressed.
>
> It may contain confidential and /or privileged material. If someone other
> than the intended recipient should receive this e-mail, he / she shall not
> be entitled to read, disseminate, disclose or duplicate it.
>
> If you receive this e-mail unintentionally, please inform us immediately
> by "reply" and then delete it from your system. Although this information
> has been compiled with great care, neither IMC Financial Markets & Asset
> Management nor any of its related entities shall accept any responsibility
> for any errors, omissions or other inaccuracies in this information or for
> the consequences thereof, nor shall it be bound in any way by the contents
> of this e-mail or its attachments. In the event of incomplete or incorrect
> transmission, please return the e-mail to the sender and permanently delete
> this message and any attachments.
>
> Messages and attachments are scanned for all known viruses. Always scan
> attachments before opening them.
>
Re: mesos in a docker container
Posted by Vinod Kone <vi...@gmail.com>.
Not sure I understand your question. From the last sentence it looks like
you already know about the "--ip" flag that could be passed to the master.
Are you looking for something else?
On Fri, Nov 8, 2013 at 1:28 PM, Khalid Goudeaux <
khalid.goudeaux@imc-chicago.com> wrote:
> Is it possible to control the ip that the mesos-master publishes to
> zookeeper for election/discovery? When running inside a docker container
> the hostname and ip that the master sees may not be the host/ip that it's
> externally accessible from.
>
> Also, inside a container the master would bind to and publish 127.0.0.1 by
> default instead of the container ip as expected. Switching to 0.0.0.0 had
> no effect. Only by explicitly specifying the container ip would it bind and
> publish a usable ip.
>
> Thanks,
> Khalid
>