You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Abel Souza <ab...@cs.umu.se> on 2018/06/12 13:12:58 UTC
mesos-execute cmd
Hello,
I believe this question relates to the framework used by the
mesos-execute command (available by default in Mesos installation):
When I request a number of cores greater than what is available in one
single node, the mesos-execute automatically turn down all offers made
by Mesos and hangs forever. E.g.: Each agent node in my cluster has 8
cores, and when I request 9 cores through mesos-execute
--resources='cpus:9', the command waits forever. But If I execute
mesos-execute --resources='cpus:8', tasks start execution right away.
So I would like to know if there is a way to enable the mesos-execute to
handle situations where multiple nodes are needed to satisfy a resource
request. If so, what would be needed?
Thank you,
/Abel Souza
Re: mesos-execute cmd
Posted by Qian Zhang <zh...@gmail.com>.
> In relation to your last comment, it does make sense to launch one task
on multiple nodes: think about fault tolerance, where one needs replicas in
case of failures in the datacenter.
The use case that you mentioned makes sense, but that is not launching one
Mesos task on multiple nodes, instead it is launching an application which
has multiple instances and each instance (as a Mesos task) is launched on a
different node (You can try Marathon which is a popular Mesos framework, it
supports this use case). In Mesos, you can never launch a single task on
multiple nodes.
Regards,
Qian Zhang
On Fri, Jun 22, 2018 at 2:30 PM, Abel Souza <ab...@cs.umu.se> wrote:
> Thank you Qian,
>
> Yes, I believe a framework is needed for handling my case. I was just
> wondering if it could be possible to simply execute one specific command in
> multiple agent nodes using current mesos-execute command.
>
> In relation to your last comment, it does make sense to launch one task on
> multiple nodes: think about fault tolerance, where one needs replicas in
> case of failures in the datacenter.
>
> Best,
>
> /Abel Souza
>
>
> On Jun 21, 2018, at 2:44 AM, Qian Zhang <zh...@gmail.com> wrote:
>
> > It is possible to use multiple offers from a single agent node to
> launch a task, but I do not think you can use multiple offers from
> different agent nodes to launch a task.
>
> Abel, this is Mesos general behavior. It does not make sense to launch a
> single task on multiple agent nodes, right? But you can write a custom
> framework to split a big resource request to multiple tasks, and launch
> each task on an agent node.
>
>
> Regards,
> Qian Zhang
>
> On Wed, Jun 20, 2018 at 10:15 PM, Tomek Janiszewski <ja...@gmail.com>
> wrote:
>
>> IMO it requires a custom framework that will handle spiting task into
>> multiple nodes. I need more details to help.
>>
>> śr., 20 cze 2018 o 15:38 użytkownik Abel Souza <ab...@cs.umu.se> napisał:
>>
>>> Anyone that could suggest me anything? Is it a problem that could be
>>> fixed by writing a custom framework?
>>>
>>> /Abel
>>>
>>> On 06/13/2018 06:05 AM, Abel Souza wrote:
>>>
>>> Did you mean through ‘mesos-execute’ command or is it a Mesos general
>>> behavior?
>>>
>>> Best,
>>>
>>> /Abel Souza
>>>
>>> On Jun 13, 2018, at 02:04, Qian Zhang <zh...@gmail.com> wrote:
>>>
>>> It is possible to use multiple offers from a single agent node to launch
>>> a task, but I do not think you can use multiple offers from different agent
>>> nodes to launch a task.
>>>
>>>
>>> Regards,
>>> Qian Zhang
>>>
>>> On Tue, Jun 12, 2018 at 9:12 PM, Abel Souza <ab...@cs.umu.se> wrote:
>>>
>>>> Hello,
>>>>
>>>> I believe this question relates to the framework used by the
>>>> mesos-execute command (available by default in Mesos installation):
>>>>
>>>> When I request a number of cores greater than what is available in one
>>>> single node, the mesos-execute automatically turn down all offers made
>>>> by Mesos and hangs forever. E.g.: Each agent node in my cluster has 8
>>>> cores, and when I request 9 cores through mesos-execute
>>>> --resources='cpus:9', the command waits forever. But If I execute mesos-execute
>>>> --resources='cpus:8', tasks start execution right away.
>>>>
>>>> So I would like to know if there is a way to enable the mesos-execute
>>>> to handle situations where multiple nodes are needed to satisfy a resource
>>>> request. If so, what would be needed?
>>>>
>>>> Thank you,
>>>>
>>>> /Abel Souza
>>>>
>>>
>>>
>>>
>
>
Re: mesos-execute cmd
Posted by Abel Souza <ab...@cs.umu.se>.
Thank you Qian,
Yes, I believe a framework is needed for handling my case. I was just wondering if it could be possible to simply execute one specific command in multiple agent nodes using current mesos-execute command.
In relation to your last comment, it does make sense to launch one task on multiple nodes: think about fault tolerance, where one needs replicas in case of failures in the datacenter.
Best,
/Abel Souza
> On Jun 21, 2018, at 2:44 AM, Qian Zhang <zh...@gmail.com> wrote:
>
> > It is possible to use multiple offers from a single agent node to launch a task, but I do not think you can use multiple offers from different agent nodes to launch a task.
>
> Abel, this is Mesos general behavior. It does not make sense to launch a single task on multiple agent nodes, right? But you can write a custom framework to split a big resource request to multiple tasks, and launch each task on an agent node.
>
>
> Regards,
> Qian Zhang
>
> On Wed, Jun 20, 2018 at 10:15 PM, Tomek Janiszewski <janiszt@gmail.com <ma...@gmail.com>> wrote:
> IMO it requires a custom framework that will handle spiting task into multiple nodes. I need more details to help.
>
> śr., 20 cze 2018 o 15:38 użytkownik Abel Souza <abel@cs.umu.se <ma...@cs.umu.se>> napisał:
> Anyone that could suggest me anything? Is it a problem that could be fixed by writing a custom framework?
>
> /Abel
>
> On 06/13/2018 06:05 AM, Abel Souza wrote:
>> Did you mean through ‘mesos-execute’ command or is it a Mesos general behavior?
>>
>> Best,
>>
>> /Abel Souza
>>
>> On Jun 13, 2018, at 02:04, Qian Zhang <zhq527725@gmail.com <ma...@gmail.com>> wrote:
>>
>>> It is possible to use multiple offers from a single agent node to launch a task, but I do not think you can use multiple offers from different agent nodes to launch a task.
>>>
>>>
>>> Regards,
>>> Qian Zhang
>>>
>>> On Tue, Jun 12, 2018 at 9:12 PM, Abel Souza <abel@cs.umu.se <ma...@cs.umu.se>> wrote:
>>> Hello,
>>>
>>> I believe this question relates to the framework used by the mesos-execute command (available by default in Mesos installation):
>>>
>>> When I request a number of cores greater than what is available in one single node, the mesos-execute automatically turn down all offers made by Mesos and hangs forever. E.g.: Each agent node in my cluster has 8 cores, and when I request 9 cores through mesos-execute --resources='cpus:9', the command waits forever. But If I execute mesos-execute --resources='cpus:8', tasks start execution right away.
>>>
>>> So I would like to know if there is a way to enable the mesos-execute to handle situations where multiple nodes are needed to satisfy a resource request. If so, what would be needed?
>>> Thank you,
>>>
>>> /Abel Souza
>>>
>
>
Re: mesos-execute cmd
Posted by Qian Zhang <zh...@gmail.com>.
> It is possible to use multiple offers from a single agent node to launch
a task, but I do not think you can use multiple offers from different agent
nodes to launch a task.
Abel, this is Mesos general behavior. It does not make sense to launch a
single task on multiple agent nodes, right? But you can write a custom
framework to split a big resource request to multiple tasks, and launch
each task on an agent node.
Regards,
Qian Zhang
On Wed, Jun 20, 2018 at 10:15 PM, Tomek Janiszewski <ja...@gmail.com>
wrote:
> IMO it requires a custom framework that will handle spiting task into
> multiple nodes. I need more details to help.
>
> śr., 20 cze 2018 o 15:38 użytkownik Abel Souza <ab...@cs.umu.se> napisał:
>
>> Anyone that could suggest me anything? Is it a problem that could be
>> fixed by writing a custom framework?
>>
>> /Abel
>>
>> On 06/13/2018 06:05 AM, Abel Souza wrote:
>>
>> Did you mean through ‘mesos-execute’ command or is it a Mesos general
>> behavior?
>>
>> Best,
>>
>> /Abel Souza
>>
>> On Jun 13, 2018, at 02:04, Qian Zhang <zh...@gmail.com> wrote:
>>
>> It is possible to use multiple offers from a single agent node to launch
>> a task, but I do not think you can use multiple offers from different agent
>> nodes to launch a task.
>>
>>
>> Regards,
>> Qian Zhang
>>
>> On Tue, Jun 12, 2018 at 9:12 PM, Abel Souza <ab...@cs.umu.se> wrote:
>>
>>> Hello,
>>>
>>> I believe this question relates to the framework used by the
>>> mesos-execute command (available by default in Mesos installation):
>>>
>>> When I request a number of cores greater than what is available in one
>>> single node, the mesos-execute automatically turn down all offers made
>>> by Mesos and hangs forever. E.g.: Each agent node in my cluster has 8
>>> cores, and when I request 9 cores through mesos-execute
>>> --resources='cpus:9', the command waits forever. But If I execute mesos-execute
>>> --resources='cpus:8', tasks start execution right away.
>>>
>>> So I would like to know if there is a way to enable the mesos-execute
>>> to handle situations where multiple nodes are needed to satisfy a resource
>>> request. If so, what would be needed?
>>>
>>> Thank you,
>>>
>>> /Abel Souza
>>>
>>
>>
>>
Re: mesos-execute cmd
Posted by Tomek Janiszewski <ja...@gmail.com>.
IMO it requires a custom framework that will handle spiting task into
multiple nodes. I need more details to help.
śr., 20 cze 2018 o 15:38 użytkownik Abel Souza <ab...@cs.umu.se> napisał:
> Anyone that could suggest me anything? Is it a problem that could be fixed
> by writing a custom framework?
>
> /Abel
>
> On 06/13/2018 06:05 AM, Abel Souza wrote:
>
> Did you mean through ‘mesos-execute’ command or is it a Mesos general
> behavior?
>
> Best,
>
> /Abel Souza
>
> On Jun 13, 2018, at 02:04, Qian Zhang <zh...@gmail.com> wrote:
>
> It is possible to use multiple offers from a single agent node to launch a
> task, but I do not think you can use multiple offers from different agent
> nodes to launch a task.
>
>
> Regards,
> Qian Zhang
>
> On Tue, Jun 12, 2018 at 9:12 PM, Abel Souza <ab...@cs.umu.se> wrote:
>
>> Hello,
>>
>> I believe this question relates to the framework used by the
>> mesos-execute command (available by default in Mesos installation):
>>
>> When I request a number of cores greater than what is available in one
>> single node, the mesos-execute automatically turn down all offers made
>> by Mesos and hangs forever. E.g.: Each agent node in my cluster has 8
>> cores, and when I request 9 cores through mesos-execute
>> --resources='cpus:9', the command waits forever. But If I execute mesos-execute
>> --resources='cpus:8', tasks start execution right away.
>>
>> So I would like to know if there is a way to enable the mesos-execute to
>> handle situations where multiple nodes are needed to satisfy a resource
>> request. If so, what would be needed?
>>
>> Thank you,
>>
>> /Abel Souza
>>
>
>
>
Re: mesos-execute cmd
Posted by Abel Souza <ab...@cs.umu.se>.
Anyone that could suggest me anything? Is it a problem that could be
fixed by writing a custom framework?
/Abel
On 06/13/2018 06:05 AM, Abel Souza wrote:
> Did you mean through ‘mesos-execute’ command or is it a Mesos general
> behavior?
>
> Best,
>
> /Abel Souza
>
> On Jun 13, 2018, at 02:04, Qian Zhang <zhq527725@gmail.com
> <ma...@gmail.com>> wrote:
>
>> It is possible to use multiple offers from a single agent node to
>> launch a task, but I do not think you can use multiple offers from
>> different agent nodes to launch a task.
>>
>>
>> Regards,
>> Qian Zhang
>>
>> On Tue, Jun 12, 2018 at 9:12 PM, Abel Souza <abel@cs.umu.se
>> <ma...@cs.umu.se>> wrote:
>>
>> Hello,
>>
>> I believe this question relates to the framework used by the
>> mesos-execute command (available by default in Mesos installation):
>>
>> When I request a number of cores greater than what is available
>> in one single node, the mesos-execute automatically turn down all
>> offers made by Mesos and hangs forever. E.g.: Each agent node in
>> my cluster has 8 cores, and when I request 9 cores through
>> mesos-execute --resources='cpus:9', the command waits forever.
>> But If I execute mesos-execute --resources='cpus:8', tasks start
>> execution right away.
>>
>> So I would like to know if there is a way to enable the
>> mesos-execute to handle situations where multiple nodes are
>> needed to satisfy a resource request. If so, what would be needed?
>>
>> Thank you,
>>
>> /Abel Souza
>>
>>
Re: mesos-execute cmd
Posted by Abel Souza <ab...@cs.umu.se>.
Did you mean through ‘mesos-execute’ command or is it a Mesos general behavior?
Best,
/Abel Souza
> On Jun 13, 2018, at 02:04, Qian Zhang <zh...@gmail.com> wrote:
>
> It is possible to use multiple offers from a single agent node to launch a task, but I do not think you can use multiple offers from different agent nodes to launch a task.
>
>
> Regards,
> Qian Zhang
>
>> On Tue, Jun 12, 2018 at 9:12 PM, Abel Souza <ab...@cs.umu.se> wrote:
>> Hello,
>>
>> I believe this question relates to the framework used by the mesos-execute command (available by default in Mesos installation):
>>
>> When I request a number of cores greater than what is available in one single node, the mesos-execute automatically turn down all offers made by Mesos and hangs forever. E.g.: Each agent node in my cluster has 8 cores, and when I request 9 cores through mesos-execute --resources='cpus:9', the command waits forever. But If I execute mesos-execute --resources='cpus:8', tasks start execution right away.
>>
>> So I would like to know if there is a way to enable the mesos-execute to handle situations where multiple nodes are needed to satisfy a resource request. If so, what would be needed?
>> Thank you,
>>
>> /Abel Souza
>
Re: mesos-execute cmd
Posted by Qian Zhang <zh...@gmail.com>.
It is possible to use multiple offers from a single agent node to launch a
task, but I do not think you can use multiple offers from different agent
nodes to launch a task.
Regards,
Qian Zhang
On Tue, Jun 12, 2018 at 9:12 PM, Abel Souza <ab...@cs.umu.se> wrote:
> Hello,
>
> I believe this question relates to the framework used by the mesos-execute
> command (available by default in Mesos installation):
>
> When I request a number of cores greater than what is available in one
> single node, the mesos-execute automatically turn down all offers made by
> Mesos and hangs forever. E.g.: Each agent node in my cluster has 8 cores,
> and when I request 9 cores through mesos-execute --resources='cpus:9',
> the command waits forever. But If I execute mesos-execute
> --resources='cpus:8', tasks start execution right away.
>
> So I would like to know if there is a way to enable the mesos-execute to
> handle situations where multiple nodes are needed to satisfy a resource
> request. If so, what would be needed?
>
> Thank you,
>
> /Abel Souza
>