You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by CCAAT <cc...@tampabay.rr.com> on 2015/01/21 21:20:07 UTC
cluster wide init
Hello all,
I was reading about Marathon: "Marathon scheduler processes were started
outside of Mesos using init, upstart, or a similar tool" [1]
So my related questions are"
Does Marathon work with mesos + Openrc as the init system?
Are there any other frameworks that work with Mesos + Openrc?
James
[1] http://mesosphere.github.io/marathon/
Re: cluster wide init
Posted by Dick Davies <di...@hellooperator.net>.
On 23 January 2015 at 21:20, Sharma Podila <sp...@netflix.com> wrote:
> Here's one possible scenario:
> A DataCenter runs Databases, Webservers, MicroServices, Hadoop or other
> batch jobs, stream processing jobs, etc. There's 1000s, if not 100s, of
> systems available for all of this. Ideally, systems running Databases are
> configured to run different services in their init than one running batch
> jobs. However, because one would want to achieve elasticity of different
> services (#systems running DBs vs. batch, for example) within a single Mesos
> cluster, Mesos would have to determine what services run on the system at
> the current time. It's like a newly installed system comes up and connects
> into Mesos and says, "hello there, I am an 8-core 48GB 1Gb eth system ready
> for service, what would you like me to do?". Mesos can choose to make it run
> any one or more of the services which would determine the set of init
> services to launch. And that may change over time as DC traffic changes.
What you're describing here is essentially the value proposition of
Mesos+marathon.
But there are still many services you need to provide in a datacenter
that aren't as
elastic as we'd like, and don't necessarily benefit from the
flexibility you're describing.
It's easy enough to lay those out with the same automation you'd use
to setup your
mesos processes under a more conventional init system.
Someone mentioned Ansible earlier, that's worked out really well in my
experience.
There's a simple Vagrant based playbook here if anyone's interested.
https://github.com/rasputnik/mesos-centos
The nice thing about Ansible is this scales up to hundreds of servers
easily, simply
by changing the inventory file.
Re: cluster wide init
Posted by Sharma Podila <sp...@netflix.com>.
(I understand this is somewhat tangential to the original email, I will
just clarify my thought in previous email. That said, this is not a fully
baked idea, just thinking out loud.)
"init" is the solution, right. However, when dealing with 1000s of systems,
one would need automation for managing init on all of them. This isn't new,
solutions exist. And I don't mean to suggest we run tasks via this
mechanism, but services that would otherwise be configured via init, for
example.
In the future, I can imagine one would need "init" or the likes to work
together with Mesos. Whether it is built into Mesos (the literal
interpretation of what I suggested) or well integrated with other solutions
with a clean interface is a design/implementation choice.
Here's one possible scenario:
A DataCenter runs Databases, Webservers, MicroServices, Hadoop or other
batch jobs, stream processing jobs, etc. There's 1000s, if not 100s, of
systems available for all of this. Ideally, systems running Databases are
configured to run different services in their init than one running batch
jobs. However, because one would want to achieve elasticity of different
services (#systems running DBs vs. batch, for example) within a single
Mesos cluster, Mesos would have to determine what services run on the
system at the current time. It's like a newly installed system comes up and
connects into Mesos and says, "hello there, I am an 8-core 48GB 1Gb eth
system ready for service, what would you like me to do?". Mesos can choose
to make it run any one or more of the services which would determine the
set of init services to launch. And that may change over time as DC traffic
changes.
On Thu, Jan 22, 2015 at 11:56 PM, Ryan Thomas <r....@gmail.com> wrote:
> If this was going to be used to allocate tasks outside of the schedulers
> resource management, and for every slave, why not just use the OS provided
> init system instead?
>
> On 22 January 2015 at 19:40, Sharma Podila <sp...@netflix.com> wrote:
>
>> Schedulers can only use resources on slaves that are unused by and
>> unallocated to other schedulers. Therefore, schedulers cannot achieve this
>> unless you "reserve" slots on every slave for the scheduler. Seems kind of
>> a forced fit. An init like support would be more fundamental to Mesos
>> cluster itself, if available.
>>
>>
>> On Thu, Jan 22, 2015 at 10:08 AM, Ryan Thomas <r....@gmail.com>
>> wrote:
>>
>>> This seems more like the responsibility of the scheduler that is
>>> running, like marathon or aurora.
>>>
>>> I haven't tried it but I would imagine if you had 10 slaves and started
>>> a job with 11 tasks with host exclusivity when you spin up an 11th slave
>>> marathon would start it there.
>>>
>>>
>>> On Thursday, 22 January 2015, Sharma Podila <sp...@netflix.com> wrote:
>>>
>>>> Just a thought looking forward...
>>>> Might be useful to define an init kind of feature in Mesos slaves.
>>>> Configuration can be defined in Mesos master that lists services that must
>>>> be run on all slaves. When slaves register, they get the list of services
>>>> to run all the time. Updates to the configuration can be dynamically
>>>> reflected on all slaves and therefore this ensures that all slaves run the
>>>> required services. Sophistication can be put in place to have different set
>>>> of services for different "types" of slaves (by resource types/quantity,
>>>> etc.).
>>>> Such a feature bodes well with Mesos being the DataCenter OS/Kernel.
>>>>
>>>>
>>>> On Thu, Jan 22, 2015 at 9:43 AM, CCAAT <cc...@tampabay.rr.com> wrote:
>>>>
>>>>> On 01/21/2015 11:10 PM, Shuai Lin wrote:
>>>>>
>>>>> OK, I'll take a look at the debian package.
>>>>>
>>>>> thanks,
>>>>> James
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> You can always write the init wrapper scripts for marathon. There is
>>>>>> an
>>>>>> official debian package, which you can find in mesos's apt repo.
>>>>>>
>>>>>> On Thu, Jan 22, 2015 at 4:20 AM, CCAAT <ccaat@tampabay.rr.com
>>>>>> <ma...@tampabay.rr.com>> wrote:
>>>>>>
>>>>>> Hello all,
>>>>>>
>>>>>> I was reading about Marathon: "Marathon scheduler processes were
>>>>>> started outside of Mesos using init, upstart, or a similar tool"
>>>>>> [1]
>>>>>>
>>>>>> This means
>>>>>>
>>>>>> So my related questions are"
>>>>>>
>>>>>> Does Marathon work with mesos + Openrc as the init system?
>>>>>>
>>>>>> Are there any other frameworks that work with Mesos + Openrc?
>>>>>>
>>>>>>
>>>>>> James
>>>>>>
>>>>>>
>>>>>>
>>>>>> [1] http://mesosphere.github.io/__marathon/
>>>>>> <http://mesosphere.github.io/marathon/>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>
>
Re: cluster wide init
Posted by Ryan Thomas <r....@gmail.com>.
If this was going to be used to allocate tasks outside of the schedulers
resource management, and for every slave, why not just use the OS provided
init system instead?
On 22 January 2015 at 19:40, Sharma Podila <sp...@netflix.com> wrote:
> Schedulers can only use resources on slaves that are unused by and
> unallocated to other schedulers. Therefore, schedulers cannot achieve this
> unless you "reserve" slots on every slave for the scheduler. Seems kind of
> a forced fit. An init like support would be more fundamental to Mesos
> cluster itself, if available.
>
>
> On Thu, Jan 22, 2015 at 10:08 AM, Ryan Thomas <r....@gmail.com>
> wrote:
>
>> This seems more like the responsibility of the scheduler that is running,
>> like marathon or aurora.
>>
>> I haven't tried it but I would imagine if you had 10 slaves and started a
>> job with 11 tasks with host exclusivity when you spin up an 11th slave
>> marathon would start it there.
>>
>>
>> On Thursday, 22 January 2015, Sharma Podila <sp...@netflix.com> wrote:
>>
>>> Just a thought looking forward...
>>> Might be useful to define an init kind of feature in Mesos slaves.
>>> Configuration can be defined in Mesos master that lists services that must
>>> be run on all slaves. When slaves register, they get the list of services
>>> to run all the time. Updates to the configuration can be dynamically
>>> reflected on all slaves and therefore this ensures that all slaves run the
>>> required services. Sophistication can be put in place to have different set
>>> of services for different "types" of slaves (by resource types/quantity,
>>> etc.).
>>> Such a feature bodes well with Mesos being the DataCenter OS/Kernel.
>>>
>>>
>>> On Thu, Jan 22, 2015 at 9:43 AM, CCAAT <cc...@tampabay.rr.com> wrote:
>>>
>>>> On 01/21/2015 11:10 PM, Shuai Lin wrote:
>>>>
>>>> OK, I'll take a look at the debian package.
>>>>
>>>> thanks,
>>>> James
>>>>
>>>>
>>>>
>>>>
>>>> You can always write the init wrapper scripts for marathon. There is an
>>>>> official debian package, which you can find in mesos's apt repo.
>>>>>
>>>>> On Thu, Jan 22, 2015 at 4:20 AM, CCAAT <ccaat@tampabay.rr.com
>>>>> <ma...@tampabay.rr.com>> wrote:
>>>>>
>>>>> Hello all,
>>>>>
>>>>> I was reading about Marathon: "Marathon scheduler processes were
>>>>> started outside of Mesos using init, upstart, or a similar tool"
>>>>> [1]
>>>>>
>>>>> This means
>>>>>
>>>>> So my related questions are"
>>>>>
>>>>> Does Marathon work with mesos + Openrc as the init system?
>>>>>
>>>>> Are there any other frameworks that work with Mesos + Openrc?
>>>>>
>>>>>
>>>>> James
>>>>>
>>>>>
>>>>>
>>>>> [1] http://mesosphere.github.io/__marathon/
>>>>> <http://mesosphere.github.io/marathon/>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>
Re: cluster wide init
Posted by Sharma Podila <sp...@netflix.com>.
Schedulers can only use resources on slaves that are unused by and
unallocated to other schedulers. Therefore, schedulers cannot achieve this
unless you "reserve" slots on every slave for the scheduler. Seems kind of
a forced fit. An init like support would be more fundamental to Mesos
cluster itself, if available.
On Thu, Jan 22, 2015 at 10:08 AM, Ryan Thomas <r....@gmail.com> wrote:
> This seems more like the responsibility of the scheduler that is running,
> like marathon or aurora.
>
> I haven't tried it but I would imagine if you had 10 slaves and started a
> job with 11 tasks with host exclusivity when you spin up an 11th slave
> marathon would start it there.
>
>
> On Thursday, 22 January 2015, Sharma Podila <sp...@netflix.com> wrote:
>
>> Just a thought looking forward...
>> Might be useful to define an init kind of feature in Mesos slaves.
>> Configuration can be defined in Mesos master that lists services that must
>> be run on all slaves. When slaves register, they get the list of services
>> to run all the time. Updates to the configuration can be dynamically
>> reflected on all slaves and therefore this ensures that all slaves run the
>> required services. Sophistication can be put in place to have different set
>> of services for different "types" of slaves (by resource types/quantity,
>> etc.).
>> Such a feature bodes well with Mesos being the DataCenter OS/Kernel.
>>
>>
>> On Thu, Jan 22, 2015 at 9:43 AM, CCAAT <cc...@tampabay.rr.com> wrote:
>>
>>> On 01/21/2015 11:10 PM, Shuai Lin wrote:
>>>
>>> OK, I'll take a look at the debian package.
>>>
>>> thanks,
>>> James
>>>
>>>
>>>
>>>
>>> You can always write the init wrapper scripts for marathon. There is an
>>>> official debian package, which you can find in mesos's apt repo.
>>>>
>>>> On Thu, Jan 22, 2015 at 4:20 AM, CCAAT <ccaat@tampabay.rr.com
>>>> <ma...@tampabay.rr.com>> wrote:
>>>>
>>>> Hello all,
>>>>
>>>> I was reading about Marathon: "Marathon scheduler processes were
>>>> started outside of Mesos using init, upstart, or a similar tool" [1]
>>>>
>>>> This means
>>>>
>>>> So my related questions are"
>>>>
>>>> Does Marathon work with mesos + Openrc as the init system?
>>>>
>>>> Are there any other frameworks that work with Mesos + Openrc?
>>>>
>>>>
>>>> James
>>>>
>>>>
>>>>
>>>> [1] http://mesosphere.github.io/__marathon/
>>>> <http://mesosphere.github.io/marathon/>
>>>>
>>>>
>>>>
>>>
>>
Re: cluster wide init
Posted by Ryan Thomas <r....@gmail.com>.
This seems more like the responsibility of the scheduler that is running,
like marathon or aurora.
I haven't tried it but I would imagine if you had 10 slaves and started a
job with 11 tasks with host exclusivity when you spin up an 11th slave
marathon would start it there.
On Thursday, 22 January 2015, Sharma Podila <sp...@netflix.com> wrote:
> Just a thought looking forward...
> Might be useful to define an init kind of feature in Mesos slaves.
> Configuration can be defined in Mesos master that lists services that must
> be run on all slaves. When slaves register, they get the list of services
> to run all the time. Updates to the configuration can be dynamically
> reflected on all slaves and therefore this ensures that all slaves run the
> required services. Sophistication can be put in place to have different set
> of services for different "types" of slaves (by resource types/quantity,
> etc.).
> Such a feature bodes well with Mesos being the DataCenter OS/Kernel.
>
>
> On Thu, Jan 22, 2015 at 9:43 AM, CCAAT <ccaat@tampabay.rr.com
> <javascript:_e(%7B%7D,'cvml','ccaat@tampabay.rr.com');>> wrote:
>
>> On 01/21/2015 11:10 PM, Shuai Lin wrote:
>>
>> OK, I'll take a look at the debian package.
>>
>> thanks,
>> James
>>
>>
>>
>>
>> You can always write the init wrapper scripts for marathon. There is an
>>> official debian package, which you can find in mesos's apt repo.
>>>
>>> On Thu, Jan 22, 2015 at 4:20 AM, CCAAT <ccaat@tampabay.rr.com
>>> <javascript:_e(%7B%7D,'cvml','ccaat@tampabay.rr.com');>
>>> <mailto:ccaat@tampabay.rr.com
>>> <javascript:_e(%7B%7D,'cvml','ccaat@tampabay.rr.com');>>> wrote:
>>>
>>> Hello all,
>>>
>>> I was reading about Marathon: "Marathon scheduler processes were
>>> started outside of Mesos using init, upstart, or a similar tool" [1]
>>>
>>> This means
>>>
>>> So my related questions are"
>>>
>>> Does Marathon work with mesos + Openrc as the init system?
>>>
>>> Are there any other frameworks that work with Mesos + Openrc?
>>>
>>>
>>> James
>>>
>>>
>>>
>>> [1] http://mesosphere.github.io/__marathon/
>>> <http://mesosphere.github.io/marathon/>
>>>
>>>
>>>
>>
>
Re: cluster wide init
Posted by CCAAT <cc...@tampabay.rr.com>.
Ah.... this makes sense, particularly over time as we add more
nodes (slaves) creating a heterogeneous mix of slaves, both in
hardware composition and it the types (function speciality) of
those slaves.
My thoughts are to use Ansible to organize, monitor and admin these
nodes directly, but, I am open to all sorts of ideas and codes
particularly if it leads to some nodes under the "systemd" umbrella
and other nodes under the openrc init system umbrella. Both working
together would provide the widest possible venue to add all sorts of
unique slave-nodes into the mesos cluster. Some of the mathematically
intensive problems we are trying to solve are most efficiently solved
with non standard types of processors (FPGA, SOC, ASIC, DSPs etc etc).
Do drop me some links on further reading, ideas and code snippets,
as they are discovered.
Thanks,
James
On 01/22/2015 12:57 PM, Sharma Podila wrote:
> Just a thought looking forward...
> Might be useful to define an init kind of feature in Mesos slaves.
> Configuration can be defined in Mesos master that lists services that
> must be run on all slaves. When slaves register, they get the list of
> services to run all the time. Updates to the configuration can be
> dynamically reflected on all slaves and therefore this ensures that all
> slaves run the required services. Sophistication can be put in place to
> have different set of services for different "types" of slaves (by
> resource types/quantity, etc.).
> Such a feature bodes well with Mesos being the DataCenter OS/Kernel.
>
>
> On Thu, Jan 22, 2015 at 9:43 AM, CCAAT <ccaat@tampabay.rr.com
> <ma...@tampabay.rr.com>> wrote:
>
> On 01/21/2015 11:10 PM, Shuai Lin wrote:
>
> OK, I'll take a look at the debian package.
>
> thanks,
> James
>
>
>
>
> You can always write the init wrapper scripts for marathon.
> There is an
> official debian package, which you can find in mesos's apt repo.
>
> On Thu, Jan 22, 2015 at 4:20 AM, CCAAT <ccaat@tampabay.rr.com
> <ma...@tampabay.rr.com>
> <mailto:ccaat@tampabay.rr.com <ma...@tampabay.rr.com>>__>
> wrote:
>
> Hello all,
>
> I was reading about Marathon: "Marathon scheduler processes
> were
> started outside of Mesos using init, upstart, or a similar
> tool" [1]
>
> This means
>
> So my related questions are"
>
> Does Marathon work with mesos + Openrc as the init system?
>
> Are there any other frameworks that work with Mesos + Openrc?
>
>
> James
>
>
>
> [1] http://mesosphere.github.io/____marathon/
> <http://mesosphere.github.io/__marathon/>
> <http://mesosphere.github.io/__marathon/
> <http://mesosphere.github.io/marathon/>>
>
>
>
>
Re: cluster wide init
Posted by Sharma Podila <sp...@netflix.com>.
Just a thought looking forward...
Might be useful to define an init kind of feature in Mesos slaves.
Configuration can be defined in Mesos master that lists services that must
be run on all slaves. When slaves register, they get the list of services
to run all the time. Updates to the configuration can be dynamically
reflected on all slaves and therefore this ensures that all slaves run the
required services. Sophistication can be put in place to have different set
of services for different "types" of slaves (by resource types/quantity,
etc.).
Such a feature bodes well with Mesos being the DataCenter OS/Kernel.
On Thu, Jan 22, 2015 at 9:43 AM, CCAAT <cc...@tampabay.rr.com> wrote:
> On 01/21/2015 11:10 PM, Shuai Lin wrote:
>
> OK, I'll take a look at the debian package.
>
> thanks,
> James
>
>
>
>
> You can always write the init wrapper scripts for marathon. There is an
>> official debian package, which you can find in mesos's apt repo.
>>
>> On Thu, Jan 22, 2015 at 4:20 AM, CCAAT <ccaat@tampabay.rr.com
>> <ma...@tampabay.rr.com>> wrote:
>>
>> Hello all,
>>
>> I was reading about Marathon: "Marathon scheduler processes were
>> started outside of Mesos using init, upstart, or a similar tool" [1]
>>
>> This means
>>
>> So my related questions are"
>>
>> Does Marathon work with mesos + Openrc as the init system?
>>
>> Are there any other frameworks that work with Mesos + Openrc?
>>
>>
>> James
>>
>>
>>
>> [1] http://mesosphere.github.io/__marathon/
>> <http://mesosphere.github.io/marathon/>
>>
>>
>>
>
Re: cluster wide init
Posted by CCAAT <cc...@tampabay.rr.com>.
On 01/21/2015 11:10 PM, Shuai Lin wrote:
OK, I'll take a look at the debian package.
thanks,
James
> You can always write the init wrapper scripts for marathon. There is an
> official debian package, which you can find in mesos's apt repo.
>
> On Thu, Jan 22, 2015 at 4:20 AM, CCAAT <ccaat@tampabay.rr.com
> <ma...@tampabay.rr.com>> wrote:
>
> Hello all,
>
> I was reading about Marathon: "Marathon scheduler processes were
> started outside of Mesos using init, upstart, or a similar tool" [1]
>
> This means
>
> So my related questions are"
>
> Does Marathon work with mesos + Openrc as the init system?
>
> Are there any other frameworks that work with Mesos + Openrc?
>
>
> James
>
>
>
> [1] http://mesosphere.github.io/__marathon/
> <http://mesosphere.github.io/marathon/>
>
>
Re: cluster wide init
Posted by Shuai Lin <li...@gmail.com>.
You can always write the init wrapper scripts for marathon. There is an
official debian package, which you can find in mesos's apt repo.
On Thu, Jan 22, 2015 at 4:20 AM, CCAAT <cc...@tampabay.rr.com> wrote:
> Hello all,
>
> I was reading about Marathon: "Marathon scheduler processes were started
> outside of Mesos using init, upstart, or a similar tool" [1]
>
This means
> So my related questions are"
>
> Does Marathon work with mesos + Openrc as the init system?
>
> Are there any other frameworks that work with Mesos + Openrc?
>
>
> James
>
>
>
> [1] http://mesosphere.github.io/marathon/
>