You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Thodoris Zois <zo...@ics.forth.gr> on 2017/07/28 08:59:51 UTC

Containerizers & Executors

Hello, 

I am trying to understand some basic components of Mesos and i have made some notes corresponding to my conclusion. If somebody could check this out and review them, or can provide me some more information about the specific topic i would be thankful. Below you will find my notes.


- Containerizers: 
    1. Mesos Containerizer
        - posix isolators 
        - cgroups isolators
    
    2. Docker containerizer
        - docker isolators
    
    3. Custom containerizer
        - my isolators
        
        
- Executors:
    Generally: Each executor has the minimum resources assigned by default (0.01 CPU & 32MB MEM)
               Executor expands its resources when a task is assigned (executor default resources + task resources)
            
    1. Mesos commandExecutor
        - run shell commands or docker
        - Each executor is a container that can have only one task to execute, you can't specify group of tasks
        - Isolation between executors/containers so isolation between tasks because each task runs in one container
        
    2. Mesos defaultExecutor
        - can run shell commands or a custom executor file e.g TestExecutor.java (from tests)
        - can execute one task per executor/container or multiple tasks (1 group).
        - No resource isolation between tasks of the same container
    
    3. Custom Executor
        - ?????
    

So, i guess i can use one offer to run some tasks on the same agent with commandExecutor or with defaultExecutor….
But how would somebody specify if the offer corresponds to one agent or multiple agents? 

Thank you,
Thodoris

Re: Containerizers & Executors

Posted by Thodoris Zois <zo...@ics.forth.gr>.
Hello,

Thank you for your response, but i am still a bit confused. 

What Mesos containerizer does? What is the point of a containerizer? 

Can commandExecutor launch a group of tasks? 

And finally is there any default executor used in tests so i can see an example? I guess that custom executor is for example TestExecutor.java. CommandExecutor is specified on taskInfo.command.commandInfo. So what about the default if i want to set? 

Thodoris



> On 31 Jul 2017, at 04:46, Vinod Kone <vi...@apache.org> wrote:
> 
> See my answers inline.
> 
>> 
>>     1. Mesos Containerizer
>>         - posix isolators
>>         - cgroups isolators
> 
> Mesos container also allows you to use custom isolators. 
> 
>  
>>     2. Docker containerizer
>>         - docker isolators
> 
> Docker containerizer doesn't have a concept of isolator(s).
> 
>  
>>     3. Custom containerizer
>>         - my isolators
> 
> It is up to the custom containerizer how it wants to do containerization; it could've have a concept of isolator or not.
>  
> 
>> - Executors:
>>     Generally: Each executor has the minimum resources assigned by default (0.01 CPU & 32MB MEM)
>>                Executor expands its resources when a task is assigned (executor default resources + task resources)
> 
> Only the built-in "default" executor needs to have a minimum amount of resources. Other built-in executors and custom executors can technically have zero resources.
> 
>  
>>     1. Mesos commandExecutor
>>         - run shell commands or docker
>>         - Each executor is a container that can have only one task to execute, you can't specify group of tasks
>>         - Isolation between executors/containers so isolation between tasks because each task runs in one container
> 
> Not that the executor that runs shell commands is called the "command" executor (run by mesos containerizer), whereas the one that runs docker images is called "docker" executor (run by docker containerizer).
>  
>  
>>     2. Mesos defaultExecutor
>>         - can run shell commands or a custom executor file e.g TestExecutor.java (from tests)
>>         - can execute one task per executor/container or multiple tasks (1 group).
>>         - No resource isolation between tasks of the same container
> 
> "default" executor is another built-in executor. it can run a group of tasks. it does not run any other (custom) executor.
> 
>  
>>     3. Custom Executor
>>         - ?????
> 
> you could write a custom executor that can run a single task or group of tasks. totally up to you.
> 
>  
>> So, i guess i can use one offer to run some tasks on the same agent with commandExecutor or with defaultExecutor….
>> But how would somebody specify if the offer corresponds to one agent or multiple agents?
> 
> Each offer has an 'AgentId' which corresponds to one agent.
> 
> HTH,
> Vinod 
>

Re: Containerizers & Executors

Posted by Vinod Kone <vi...@apache.org>.
See my answers inline.


>     1. Mesos Containerizer
>         - posix isolators
>         - cgroups isolators
>

Mesos container also allows you to use custom isolators.



>     2. Docker containerizer
>         - docker isolators
>

Docker containerizer doesn't have a concept of isolator(s).



>     3. Custom containerizer
>         - my isolators
>

It is up to the custom containerizer how it wants to do containerization;
it could've have a concept of isolator or not.


- Executors:
>     Generally: Each executor has the minimum resources assigned by default
> (0.01 CPU & 32MB MEM)
>                Executor expands its resources when a task is assigned
> (executor default resources + task resources)
>

Only the built-in "default" executor needs to have a minimum amount of
resources. Other built-in executors and custom executors can technically
have zero resources.



>     1. Mesos commandExecutor
>         - run shell commands or docker
>         - Each executor is a container that can have only one task to
> execute, you can't specify group of tasks
>         - Isolation between executors/containers so isolation between
> tasks because each task runs in one container
>

Not that the executor that runs shell commands is called the "command"
executor (run by mesos containerizer), whereas the one that runs docker
images is called "docker" executor (run by docker containerizer).



>     2. Mesos defaultExecutor
>         - can run shell commands or a custom executor file e.g
> TestExecutor.java (from tests)
>         - can execute one task per executor/container or multiple tasks (1
> group).
>         - No resource isolation between tasks of the same container
>

"default" executor is another built-in executor. it can run a group of
tasks. it does not run any other (custom) executor.



>     3. Custom Executor
>         - ?????
>

you could write a custom executor that can run a single task or group of
tasks. totally up to you.



> So, i guess i can use one offer to run some tasks on the same agent with
> commandExecutor or with defaultExecutor….
> But how would somebody specify if the offer corresponds to one agent or
> multiple agents?
>

Each offer has an 'AgentId' which corresponds to one agent.

HTH,
Vinod