Re: Questions on task resource specs

[Benjamin Mahler <be...@gmail.com>]

On Fri, Aug 30, 2013 at 7:43 AM, Li Jin <ic...@gmail.com> wrote:

> Let me explain more to see if this makes sense.
>
> My tasks are processes, they take cpu/mem. I have a existing system that
> essentially do two things (1) launch tasks unshared host used by a single
> user (2) launch tasks to shared host used by multiple users. Mesos makes a
> lot of sense in case (2) so I am working on reimplementing the system on
> top of Mesos. In case (1) however, most users want the ability to not to
> think about resources, it's their host and their just want to run some
> commands on it. 99% of the time resource is not the issue since the users
> know what they are doing. I am trying to keep the same behavior. Providing
> sensible default is not perfect for this case either because say I put
> cpu=1 as default, then a host can only run 12 or 24 tasks at the same time.
> They may be running 50 just fine now.
>

Perhaps in this case you simply want to use process isolation on those
slaves. Another option for you here is to use cgroups isolation but not
with the cpu and memory subsystems (it is supposed to work, but there are a
few spots in CgroupsIsolator that assume 'cpu' and 'mem' are attached, so
that would need fixing first!).


> I am trying to enjoy the benefit of Mesos's resource management for case
> (2) while not breaking case (1) . If later some user in case (1) want to
> use resource management feature, I can make them specify resource for their
> jobs. This is an easier migration path. That's why I come up with the
> cpu=0.0001 default with process isolation.
>
> Btw, does Mesos support usage of different isolation modules in different
> slaves at the same time?
>

Yes. But the isolation being used is not part of the information being sent
to your scheduler, so if you're looking to treat them differently then you
will likely want to set attributes on the slaves using --attributes. For
example, you could mark your slaves as follows:

Process Isolation
./mesos-slave ... --isolation=process --attributes=isolation:process

Cgroups Isolation
./mesos-slave ... --isolation=process --attributes=isolation:cgroups

Then your scheduler will be able to differentiate these slaves using the
attributes present in their SlaveInfo.


>
> Thanks,
> Li
>
>
>
>
> On Thu, Aug 29, 2013 at 9:07 PM, Vinod Kone <vi...@gmail.com> wrote:
>
>> Also, can you explain a bit more on what your "tasks" are? Since tasks
>> are essentially units of work, it seems odd that you want to have tasks
>> that do not take up any cpu/mem. If a task is a unix process, it has to use
>> some cpu/mem.
>>
>>
>> On Thu, Aug 29, 2013 at 8:09 PM, Benjamin Mahler <
>> benjamin.mahler@gmail.com> wrote:
>>
>>> Does that mean your framework only works correctly with slaves using
>>> process isolation? With the technique you mentioned, the tasks will get
>>> very little cpu when cgroups isolation is running on the slave. Perhaps it
>>> would be better to choose some sensible defaults for cpu and memory?
>>>
>>>
>>> On Thu, Aug 29, 2013 at 2:51 PM, Li Jin <ic...@gmail.com> wrote:
>>>
>>>> I see. Things are easier when the users can always specify cpu and mem.
>>>> However, one use case I would like to support in my framework is to allow
>>>> users to not specify cpu and mem, and just use the process isolation. I am
>>>> wondering if this is a supported use case. One way to make it work now is
>>>> to use somethig like cpu=0.00001 but that feels pretty hacky.
>>>>
>>>> Li
>>>>
>>>>
>>>> On Thu, Aug 29, 2013 at 5:13 PM, Benjamin Hindman <
>>>> benjamin.hindman@gmail.com> wrote:
>>>>
>>>>> Resources like cpu and memory are "grandfathered in" as first-class
>>>>> resources. We won't be adding any other resource values like that unless
>>>>> the community felt otherwise. Instead, as we add new resources we'll have
>>>>> sensible defaults so your framework won't break by you might not get the
>>>>> performance you want unless you explicitly set the resource value yourself.
>>>>> But this should at least give you a sensible path to upgrade your framework!
>>>>>
>>>>>
>>>>> On Thu, Aug 29, 2013 at 1:47 PM, Li Jin <ic...@gmail.com> wrote:
>>>>>
>>>>>> Ben,
>>>>>>
>>>>>> Thanks for the reply. So my understanding is if some isolation module
>>>>>> (in this case cgroups) needs certain resource values to be specified, I
>>>>>> would need to specify those even if I am using a different isolation
>>>>>> module, is that right? I am a bit worried that future releases might
>>>>>> require other resource values to be specified that would break my framework.
>>>>>>
>>>>>> Li
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 29, 2013 at 3:10 PM, Benjamin Hindman <
>>>>>> benjamin.hindman@gmail.com> wrote:
>>>>>>
>>>>>>> When we attempt to isolate the resources via control groups on Linux
>>>>>>> we need at least some value. That being said, isolation is coarser grained
>>>>>>> than what you can specify with a double, and 0.0001 gets rounded to a
>>>>>>> different value (via cpu shares). I could imagine doing validation on cpu
>>>>>>> resources to not allow something below the granularity that we can actually
>>>>>>> isolate. How does that sound?
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 29, 2013 at 8:58 AM, Li Jin <ic...@gmail.com>wrote:
>>>>>>>
>>>>>>>> Hello guys,
>>>>>>>>
>>>>>>>> I have observed this behavior:
>>>>>>>>
>>>>>>>> (1) If I launchTasks with a task with no resource spec, mesos says
>>>>>>>> "Error validating task 6901953 : Task uses no resources"
>>>>>>>> (2) If I launchTasks with a task with cpu=0.00001 and no other
>>>>>>>> resource spec, mesos launches the task
>>>>>>>>
>>>>>>>> These two seem to be practically the same, I am wondering what's
>>>>>>>> the rationale to have different behavior for these two cases.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Li
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>