You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Tomek Janiszewski <ja...@gmail.com> on 2017/01/16 11:41:18 UTC
Default executor grace period
Hi
I tried to use grace period with default Mesos executor. I assumed it works
as follow:
1. Start command: sh -c "command ..."
2. Sent SIGSTOP to process tree: sh, command
3. Sent SIGTERM to process tree: sh, command
4. Wait for processes to finish or grace period to elapse
5. sh finish while command could be still running and attached to init
6. Sent SIGKILL to process tree: command
I notice that SIGKILL is not sent and executor finished when sh returns.
When Mesos is running with POSIX contenerizer this leads command to live
forever (if it ignores SIGTERM). When contenerizer is used command is
killed when it's container is destroyed.
Is this desired behavior? How to use grace period with default executor?
Thanks
Tomek
Re: Default executor grace period
Posted by Alex Rukletsov <al...@mesosphere.com>.
Commented on the ticket.
On Tue, Jan 17, 2017 at 12:27 PM, Tomek Janiszewski <ja...@gmail.com>
wrote:
> Created issue for this: https://issues.apache.org/jira/browse/MESOS-6933
>
> pon., 16 sty 2017 o 17:13 użytkownik Tomek Janiszewski <ja...@gmail.com>
> napisał:
>
>> I looks like it's supported because executor prints grace period[1]. On
>> the other hand executor launches sh that launch command and shell executes
>> faster then command after receiving SIGTERM. Causing process to be attached
>> to init and leaked. In my opinion default executor should not sent SIGTERM
>> to sh but only to its children. This will allow proper escalation to
>> SIGKILL because sh will leave as long its children are alive.
>>
>> 1: https://github.com/apache/mesos/blob/c4667d6f1b49d30089e6cb5874b673
>> 7a9bd3f044/src/launcher/executor.cpp#L479-L480
>>
>> pon., 16 sty 2017 o 16:35 użytkownik haosdent <ha...@gmail.com>
>> napisał:
>>
>> It looks like default-executor have not yet handle
>> `--executor_shutdown_grace_period`。
>>
>> On Mon, Jan 16, 2017 at 7:41 PM, Tomek Janiszewski <ja...@gmail.com>
>> wrote:
>>
>> Hi
>>
>> I tried to use grace period with default Mesos executor. I assumed it
>> works as follow:
>>
>> 1. Start command: sh -c "command ..."
>> 2. Sent SIGSTOP to process tree: sh, command
>> 3. Sent SIGTERM to process tree: sh, command
>> 4. Wait for processes to finish or grace period to elapse
>> 5. sh finish while command could be still running and attached to init
>> 6. Sent SIGKILL to process tree: command
>>
>> I notice that SIGKILL is not sent and executor finished when sh returns.
>> When Mesos is running with POSIX contenerizer this leads command to live
>> forever (if it ignores SIGTERM). When contenerizer is used command is
>> killed when it's container is destroyed.
>>
>> Is this desired behavior? How to use grace period with default executor?
>>
>> Thanks
>> Tomek
>>
>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>>
Re: Default executor grace period
Posted by Tomek Janiszewski <ja...@gmail.com>.
Created issue for this: https://issues.apache.org/jira/browse/MESOS-6933
pon., 16 sty 2017 o 17:13 użytkownik Tomek Janiszewski <ja...@gmail.com>
napisał:
> I looks like it's supported because executor prints grace period[1]. On
> the other hand executor launches sh that launch command and shell executes
> faster then command after receiving SIGTERM. Causing process to be attached
> to init and leaked. In my opinion default executor should not sent SIGTERM
> to sh but only to its children. This will allow proper escalation to
> SIGKILL because sh will leave as long its children are alive.
>
> 1:
> https://github.com/apache/mesos/blob/c4667d6f1b49d30089e6cb5874b6737a9bd3f044/src/launcher/executor.cpp#L479-L480
>
> pon., 16 sty 2017 o 16:35 użytkownik haosdent <ha...@gmail.com>
> napisał:
>
> It looks like default-executor have not yet handle
> `--executor_shutdown_grace_period`。
>
> On Mon, Jan 16, 2017 at 7:41 PM, Tomek Janiszewski <ja...@gmail.com>
> wrote:
>
> Hi
>
> I tried to use grace period with default Mesos executor. I assumed it
> works as follow:
>
> 1. Start command: sh -c "command ..."
> 2. Sent SIGSTOP to process tree: sh, command
> 3. Sent SIGTERM to process tree: sh, command
> 4. Wait for processes to finish or grace period to elapse
> 5. sh finish while command could be still running and attached to init
> 6. Sent SIGKILL to process tree: command
>
> I notice that SIGKILL is not sent and executor finished when sh returns.
> When Mesos is running with POSIX contenerizer this leads command to live
> forever (if it ignores SIGTERM). When contenerizer is used command is
> killed when it's container is destroyed.
>
> Is this desired behavior? How to use grace period with default executor?
>
> Thanks
> Tomek
>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>
>
Re: Default executor grace period
Posted by Tomek Janiszewski <ja...@gmail.com>.
I looks like it's supported because executor prints grace period[1]. On the
other hand executor launches sh that launch command and shell executes
faster then command after receiving SIGTERM. Causing process to be attached
to init and leaked. In my opinion default executor should not sent SIGTERM
to sh but only to its children. This will allow proper escalation to
SIGKILL because sh will leave as long its children are alive.
1:
https://github.com/apache/mesos/blob/c4667d6f1b49d30089e6cb5874b6737a9bd3f044/src/launcher/executor.cpp#L479-L480
pon., 16 sty 2017 o 16:35 użytkownik haosdent <ha...@gmail.com> napisał:
> It looks like default-executor have not yet handle
> `--executor_shutdown_grace_period`。
>
> On Mon, Jan 16, 2017 at 7:41 PM, Tomek Janiszewski <ja...@gmail.com>
> wrote:
>
> Hi
>
> I tried to use grace period with default Mesos executor. I assumed it
> works as follow:
>
> 1. Start command: sh -c "command ..."
> 2. Sent SIGSTOP to process tree: sh, command
> 3. Sent SIGTERM to process tree: sh, command
> 4. Wait for processes to finish or grace period to elapse
> 5. sh finish while command could be still running and attached to init
> 6. Sent SIGKILL to process tree: command
>
> I notice that SIGKILL is not sent and executor finished when sh returns.
> When Mesos is running with POSIX contenerizer this leads command to live
> forever (if it ignores SIGTERM). When contenerizer is used command is
> killed when it's container is destroyed.
>
> Is this desired behavior? How to use grace period with default executor?
>
> Thanks
> Tomek
>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>
Re: Default executor grace period
Posted by haosdent <ha...@gmail.com>.
It looks like default-executor have not yet handle
`--executor_shutdown_grace_period`。
On Mon, Jan 16, 2017 at 7:41 PM, Tomek Janiszewski <ja...@gmail.com>
wrote:
> Hi
>
> I tried to use grace period with default Mesos executor. I assumed it
> works as follow:
>
> 1. Start command: sh -c "command ..."
> 2. Sent SIGSTOP to process tree: sh, command
> 3. Sent SIGTERM to process tree: sh, command
> 4. Wait for processes to finish or grace period to elapse
> 5. sh finish while command could be still running and attached to init
> 6. Sent SIGKILL to process tree: command
>
> I notice that SIGKILL is not sent and executor finished when sh returns.
> When Mesos is running with POSIX contenerizer this leads command to live
> forever (if it ignores SIGTERM). When contenerizer is used command is
> killed when it's container is destroyed.
>
> Is this desired behavior? How to use grace period with default executor?
>
> Thanks
> Tomek
>
--
Best Regards,
Haosdent Huang