You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Zhitao Li <zh...@gmail.com> on 2018/06/15 18:06:13 UTC
Re: narrowing task sandbox permissions
Sorry for getting back to this really late, but we got bit by this behavior
change in our environment.
The broken scenario we had:
1. We are using Aurora to launch docker containerizer based tasks on
Mesos;
2. Most of our docker containers had some legacy behavior: *the
execution entered as "root" in the entry point script,* setup a couple
of symlinks and other preparation work, then *"de-escalate" into a non
privileged user (i.e, "user")*;
1. This was added so that the entry point script has enough
permission to reconfigure certain side car processes (i.e, nginx) and
filesystem paths;
3. unfortunately, the "user" user will lose access to the sandbox after
this change.
While I'd acknowledge that above behavior is legacy and a piece of major
tech debt, cleaning it up for the thousands of applications on our platform
was never easy. Given that our org has other useful features available in
1.6, I would propose a couple of options:
1. making the sandbox permission bits configurable
1. Certain framework knows that their tasks do not leave sensitive
data on sandbox so we could provide this flexibility (it's very useful in
practice for migration to a container based system);
2. Alternatively, making this possible to reconfigure on agent flags:
This could be more secure and easier to manage, but lacks flexibility of
allowing different frameworks to do different things.
2. Until the customization is in place, consider a revert of the
permission bit change so we preserve the original behavior.
Thanks.
Zhitao and Jason
Re: narrowing task sandbox permissions
Posted by Zhitao Li <zh...@gmail.com>.
Adding James directly.
On Fri, Jun 15, 2018 at 11:06 AM Zhitao Li <zh...@gmail.com> wrote:
> Sorry for getting back to this really late, but we got bit by this
> behavior change in our environment.
>
> The broken scenario we had:
>
> 1. We are using Aurora to launch docker containerizer based tasks on
> Mesos;
> 2. Most of our docker containers had some legacy behavior: *the
> execution entered as "root" in the entry point script,* setup a couple
> of symlinks and other preparation work, then *"de-escalate" into a non
> privileged user (i.e, "user")*;
> 1. This was added so that the entry point script has enough
> permission to reconfigure certain side car processes (i.e, nginx) and
> filesystem paths;
> 3. unfortunately, the "user" user will lose access to the sandbox
> after this change.
>
>
> While I'd acknowledge that above behavior is legacy and a piece of major
> tech debt, cleaning it up for the thousands of applications on our platform
> was never easy. Given that our org has other useful features available in
> 1.6, I would propose a couple of options:
>
> 1. making the sandbox permission bits configurable
> 1. Certain framework knows that their tasks do not leave sensitive
> data on sandbox so we could provide this flexibility (it's very useful in
> practice for migration to a container based system);
> 2. Alternatively, making this possible to reconfigure on agent
> flags: This could be more secure and easier to manage, but lacks
> flexibility of allowing different frameworks to do different things.
> 2. Until the customization is in place, consider a revert of the
> permission bit change so we preserve the original behavior.
>
> Thanks.
>
> Zhitao and Jason
>
--
Cheers,
Zhitao Li
Re: narrowing task sandbox permissions
Posted by James Peach <jo...@gmail.com>.
> On Jun 15, 2018, at 11:06 AM, Zhitao Li <zh...@gmail.com> wrote:
>
> Sorry for getting back to this really late, but we got bit by this behavior
> change in our environment.
>
> The broken scenario we had:
>
> 1. We are using Aurora to launch docker containerizer based tasks on
> Mesos;
> 2. Most of our docker containers had some legacy behavior: *the
> execution entered as "root" in the entry point script,* setup a couple
> of symlinks and other preparation work, then *"de-escalate" into a non
> privileged user (i.e, "user")*;
> 1. This was added so that the entry point script has enough
> permission to reconfigure certain side car processes (i.e, nginx) and
> filesystem paths;
> 3. unfortunately, the "user" user will lose access to the sandbox after
> this change.
>
>
> While I'd acknowledge that above behavior is legacy and a piece of major
> tech debt, cleaning it up for the thousands of applications on our platform
> was never easy. Given that our org has other useful features available in
> 1.6, I would propose a couple of options:
>
> 1. making the sandbox permission bits configurable
> 1. Certain framework knows that their tasks do not leave sensitive
> data on sandbox so we could provide this flexibility (it's very useful in
> practice for migration to a container based system);
> 2. Alternatively, making this possible to reconfigure on agent flags:
> This could be more secure and easier to manage, but lacks flexibility of
> allowing different frameworks to do different things.
> 2. Until the customization is in place, consider a revert of the
> permission bit change so we preserve the original behavior.
That's a pretty unfortunate outcome. Can you change the permissions in your script, or happy a Mesos patch until the legacy can be addressed?
J
Re: narrowing task sandbox permissions
Posted by Zhitao Li <zh...@gmail.com>.
Adding James directly.
On Fri, Jun 15, 2018 at 11:06 AM Zhitao Li <zh...@gmail.com> wrote:
> Sorry for getting back to this really late, but we got bit by this
> behavior change in our environment.
>
> The broken scenario we had:
>
> 1. We are using Aurora to launch docker containerizer based tasks on
> Mesos;
> 2. Most of our docker containers had some legacy behavior: *the
> execution entered as "root" in the entry point script,* setup a couple
> of symlinks and other preparation work, then *"de-escalate" into a non
> privileged user (i.e, "user")*;
> 1. This was added so that the entry point script has enough
> permission to reconfigure certain side car processes (i.e, nginx) and
> filesystem paths;
> 3. unfortunately, the "user" user will lose access to the sandbox
> after this change.
>
>
> While I'd acknowledge that above behavior is legacy and a piece of major
> tech debt, cleaning it up for the thousands of applications on our platform
> was never easy. Given that our org has other useful features available in
> 1.6, I would propose a couple of options:
>
> 1. making the sandbox permission bits configurable
> 1. Certain framework knows that their tasks do not leave sensitive
> data on sandbox so we could provide this flexibility (it's very useful in
> practice for migration to a container based system);
> 2. Alternatively, making this possible to reconfigure on agent
> flags: This could be more secure and easier to manage, but lacks
> flexibility of allowing different frameworks to do different things.
> 2. Until the customization is in place, consider a revert of the
> permission bit change so we preserve the original behavior.
>
> Thanks.
>
> Zhitao and Jason
>
--
Cheers,
Zhitao Li
Re: narrowing task sandbox permissions
Posted by James Peach <jo...@gmail.com>.
> On Jun 15, 2018, at 11:06 AM, Zhitao Li <zh...@gmail.com> wrote:
>
> Sorry for getting back to this really late, but we got bit by this behavior
> change in our environment.
>
> The broken scenario we had:
>
> 1. We are using Aurora to launch docker containerizer based tasks on
> Mesos;
> 2. Most of our docker containers had some legacy behavior: *the
> execution entered as "root" in the entry point script,* setup a couple
> of symlinks and other preparation work, then *"de-escalate" into a non
> privileged user (i.e, "user")*;
> 1. This was added so that the entry point script has enough
> permission to reconfigure certain side car processes (i.e, nginx) and
> filesystem paths;
> 3. unfortunately, the "user" user will lose access to the sandbox after
> this change.
>
>
> While I'd acknowledge that above behavior is legacy and a piece of major
> tech debt, cleaning it up for the thousands of applications on our platform
> was never easy. Given that our org has other useful features available in
> 1.6, I would propose a couple of options:
>
> 1. making the sandbox permission bits configurable
> 1. Certain framework knows that their tasks do not leave sensitive
> data on sandbox so we could provide this flexibility (it's very useful in
> practice for migration to a container based system);
> 2. Alternatively, making this possible to reconfigure on agent flags:
> This could be more secure and easier to manage, but lacks flexibility of
> allowing different frameworks to do different things.
> 2. Until the customization is in place, consider a revert of the
> permission bit change so we preserve the original behavior.
That's a pretty unfortunate outcome. Can you change the permissions in your script, or happy a Mesos patch until the legacy can be addressed?
J