You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Gilbert Song <gi...@mesosphere.io> on 2016/03/15 18:48:19 UTC
[DISCUSS] Fetching Docker Images Requiring User Credentials.
Hi folks,
We want to raise a discussion here, seeking suggestions about passing
credentials in a secure way. This relates to the JIRA MESOS-4938
<https://issues.apache.org/jira/browse/MESOS-4938>, supporting docker
private registry authentication in unified containerizer. In fact, this
problem is not limited to docker registry. For instance, how can we support
CommandInfo.URIs that need credentials?
For the docker registry problem, credentials have to be included when
communicating with the docker auth server. We have two options here:
Option 1: Passing credentials in protobuf Image::Docker.
Pros: This means supporting per-container docker registry, which is robust
because different registry can be reached by an agent, configurable by
users.
Cons: So SSL has to be enabled to encrypt the communication between master
and slave to prevent credentials from being seen by others. We also need to
make sure we don’t expose credentials in any endpoint.
Option 2: Passing credentials as an agent flag.
Pros: Not necessary to be SSL enabled.
Cons: No per-container registry support (imagine a multi-tenant cluster).
Some background: How does docker containerizer solve this issue?
In docker containerizer, we ask the framework to specify a URI for their
task/executor that points to the .dockercfg(now ~/.docker/config.json)
which contains the user and password information. The .dockercfg will be
saved in the sandbox by the fetcher. When we call docker pull, we set the
$HOME env for the subprocess to point to the sandbox so that the docker
client can pick up that .dockercfg when pulling images.
Any comment/advice will be absolutely welcome!
Thanks,
Gilbert/Jie
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Avinash Sridharan <av...@mesosphere.io>.
This might be a bit far fetched, but does it make sense to associate these
credential configurations with roles? Roles identify the capability of
frameworks in obtaining resources so was wondering if we can use the same
capability to distinguish (and control access) to credentials ?
On Wed, Mar 16, 2016 at 1:34 PM, Kevin Klues <kl...@gmail.com> wrote:
> On Tue, Mar 15, 2016 at 6:10 PM, Gilbert Song <gi...@mesosphere.io>
> wrote:
> > @Kevin, thanks for writing it down in detail. It sounds good that a more
> > concrete
> > schema is designed to generally solve similar auth problem.
> >
> > Just have two potential issues inlined below:
> >
> > On Tue, Mar 15, 2016 at 5:39 PM, Kevin Klues <kl...@gmail.com> wrote:
> >>
> >> Yeah, option 2.
> >>
> >> I was trying to expand on Avinash's suggestion and make it a bit more
> >> concrete in terms of what was being proposed. Needing to reload the
> >> agent just to update the list of credentials it accepts seems
> >> undesirable though.
> >>
> >> Maybe we could have a way to start the agent with a default config (by
> >> iterating on the schema from my previous email), but allow newly
> >> launched frameworks to somehow update the config on the fly through a
> >
> >
> > Will it be too expensive to update all agents every time a new framework
> > joins (handling consensus problem as well)?
>
> Not sure, I haven't though about it in depth. What I was picturing
> though was something exactly like what you describe for how the docker
> containerizer currently solves this problem, except instead of using
> docker/config.json directly, use a new credentials.json file which
> follows a schema similar to what I proposed above.
>
> >>
> >> file in their sandbox that follows the same schema.
> >
> >
> > Does that mean the file in sandbox should be exposed to each other?
> >
> >>
> >> On Tue, Mar 15, 2016 at 5:25 PM, Jie Yu <yu...@gmail.com> wrote:
> >> > Kevin, are you suggesting option 2 and having a config file like the
> >> > above?
> >> >
> >> > I think another downside of a per-agent config is that it's hard to
> >> > maintain this. What if a new framework joins and has a new credential
> >> > for
> >> > the docker images. Do we need to restart the agent to reload the
> config?
> >> >
> >> > - Jie
> >> >
> >> > On Tue, Mar 15, 2016 at 1:25 PM, Kevin Klues <kl...@gmail.com>
> wrote:
> >> >
> >> >> Can we be a bit more concrete here and try to build up a schema for
> >> >> this.
> >> >> Maybe something like:
> >> >>
> >> >> {
> >> >> [
> >> >> {
> >> >> "service" : "docker",
> >> >> "registries" :
> >> >> [
> >> >> "uri" : "<uri>",
> >> >> "default_credentials" :
> >> >> {
> >> >> "type" : "<type>",
> >> >> "credential" :
> >> >> {
> >> >> // Custom based on type...
> >> >> }
> >> >> },
> >> >> "image_credentials" :
> >> >> [
> >> >> {
> >> >> "image_name" : "<image_name>",
> >> >> "type" : "<type>",
> >> >> "credential" :
> >> >> {
> >> >> // Custom based on type...
> >> >> },
> >> >> },
> >> >> ...
> >> >> ],
> >> >> ...
> >> >> ]
> >> >> ...
> >> >> },
> >> >> ...
> >> >> ]
> >> >> }
> >> >>
> >> >>
> >> >> On Tue, Mar 15, 2016 at 12:57 PM, Jie Yu <yu...@gmail.com>
> wrote:
> >> >> >>
> >> >> >> Yeah I was thinking having the JSON as a dictionary with keys
> being
> >> >> >> the
> >> >> >> registry URI (appc/docker) and the values being credentials (which
> >> >> >> will
> >> >> be
> >> >> >> a dictionary as well I guess).
> >> >> >
> >> >> >
> >> >> > Using registry URI as the key is problematic. Think about the
> public
> >> >> docker
> >> >> > hub. Different frameworks might want to use different credentials
> to
> >> >> access
> >> >> > their docker images.
> >> >> >
> >> >> > - Jie
> >> >> >
> >> >> > On Tue, Mar 15, 2016 at 11:52 AM, Avinash Sridharan <
> >> >> avinash@mesosphere.io
> >> >> >
> >> >> > wrote:
> >> >> >
> >> >> >> On Tue, Mar 15, 2016 at 11:43 AM, Vinod Kone <
> vinodkone@apache.org>
> >> >> wrote:
> >> >> >>
> >> >> >> > moved core@ to *bcc*
> >> >> >> >
> >> >> >> > On Tue, Mar 15, 2016 at 11:18 AM, Avinash Sridharan <
> >> >> >> avinash@mesosphere.io
> >> >> >> > > wrote:
> >> >> >> >
> >> >> >> >> Why not follow option 2, but instead of passing the agent
> >> >> credentials,
> >> >> >> >> pass a location to the flag where credentials for the registry
> >> >> >> >> can be
> >> >> >> found
> >> >> >> >> (in JSON)? The frameworks can set credentials (maybe registry
> >> >> >> >> name or
> >> >> >> URL
> >> >> >> >> to the registry), and the credentials can be learnt from the
> JSON
> >> >> >> config.
> >> >> >> >>
> >> >> >> >
> >> >> >> > What if we need credentials for multiple-registries? Have a JSON
> >> >> >> > with
> >> >> one
> >> >> >> > credential per registry I guess? But if possible, I would love
> to
> >> >> solve
> >> >> >> > this more generally as possible; as Gilbert mentioned, this is
> not
> >> >> >> > a
> >> >> >> > problem just for Docker images but any URIs that need AuthN.
> >> >> >> >
> >> >> >> Yeah I was thinking having the JSON as a dictionary with keys
> being
> >> >> >> the
> >> >> >> registry URI (appc/docker) and the values being credentials (which
> >> >> >> will
> >> >> be
> >> >> >> a dictionary as well I guess).
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Avinash Sridharan, Mesosphere
> >> >> >> +1 (323) 702 5245
> >> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> ~Kevin
> >> >>
> >>
> >>
> >>
> >> --
> >> ~Kevin
> >
> >
>
>
>
> --
> ~Kevin
>
--
Avinash Sridharan, Mesosphere
+1 (323) 702 5245
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Kevin Klues <kl...@gmail.com>.
On Tue, Mar 15, 2016 at 6:10 PM, Gilbert Song <gi...@mesosphere.io> wrote:
> @Kevin, thanks for writing it down in detail. It sounds good that a more
> concrete
> schema is designed to generally solve similar auth problem.
>
> Just have two potential issues inlined below:
>
> On Tue, Mar 15, 2016 at 5:39 PM, Kevin Klues <kl...@gmail.com> wrote:
>>
>> Yeah, option 2.
>>
>> I was trying to expand on Avinash's suggestion and make it a bit more
>> concrete in terms of what was being proposed. Needing to reload the
>> agent just to update the list of credentials it accepts seems
>> undesirable though.
>>
>> Maybe we could have a way to start the agent with a default config (by
>> iterating on the schema from my previous email), but allow newly
>> launched frameworks to somehow update the config on the fly through a
>
>
> Will it be too expensive to update all agents every time a new framework
> joins (handling consensus problem as well)?
Not sure, I haven't though about it in depth. What I was picturing
though was something exactly like what you describe for how the docker
containerizer currently solves this problem, except instead of using
docker/config.json directly, use a new credentials.json file which
follows a schema similar to what I proposed above.
>>
>> file in their sandbox that follows the same schema.
>
>
> Does that mean the file in sandbox should be exposed to each other?
>
>>
>> On Tue, Mar 15, 2016 at 5:25 PM, Jie Yu <yu...@gmail.com> wrote:
>> > Kevin, are you suggesting option 2 and having a config file like the
>> > above?
>> >
>> > I think another downside of a per-agent config is that it's hard to
>> > maintain this. What if a new framework joins and has a new credential
>> > for
>> > the docker images. Do we need to restart the agent to reload the config?
>> >
>> > - Jie
>> >
>> > On Tue, Mar 15, 2016 at 1:25 PM, Kevin Klues <kl...@gmail.com> wrote:
>> >
>> >> Can we be a bit more concrete here and try to build up a schema for
>> >> this.
>> >> Maybe something like:
>> >>
>> >> {
>> >> [
>> >> {
>> >> "service" : "docker",
>> >> "registries" :
>> >> [
>> >> "uri" : "<uri>",
>> >> "default_credentials" :
>> >> {
>> >> "type" : "<type>",
>> >> "credential" :
>> >> {
>> >> // Custom based on type...
>> >> }
>> >> },
>> >> "image_credentials" :
>> >> [
>> >> {
>> >> "image_name" : "<image_name>",
>> >> "type" : "<type>",
>> >> "credential" :
>> >> {
>> >> // Custom based on type...
>> >> },
>> >> },
>> >> ...
>> >> ],
>> >> ...
>> >> ]
>> >> ...
>> >> },
>> >> ...
>> >> ]
>> >> }
>> >>
>> >>
>> >> On Tue, Mar 15, 2016 at 12:57 PM, Jie Yu <yu...@gmail.com> wrote:
>> >> >>
>> >> >> Yeah I was thinking having the JSON as a dictionary with keys being
>> >> >> the
>> >> >> registry URI (appc/docker) and the values being credentials (which
>> >> >> will
>> >> be
>> >> >> a dictionary as well I guess).
>> >> >
>> >> >
>> >> > Using registry URI as the key is problematic. Think about the public
>> >> docker
>> >> > hub. Different frameworks might want to use different credentials to
>> >> access
>> >> > their docker images.
>> >> >
>> >> > - Jie
>> >> >
>> >> > On Tue, Mar 15, 2016 at 11:52 AM, Avinash Sridharan <
>> >> avinash@mesosphere.io
>> >> >
>> >> > wrote:
>> >> >
>> >> >> On Tue, Mar 15, 2016 at 11:43 AM, Vinod Kone <vi...@apache.org>
>> >> wrote:
>> >> >>
>> >> >> > moved core@ to *bcc*
>> >> >> >
>> >> >> > On Tue, Mar 15, 2016 at 11:18 AM, Avinash Sridharan <
>> >> >> avinash@mesosphere.io
>> >> >> > > wrote:
>> >> >> >
>> >> >> >> Why not follow option 2, but instead of passing the agent
>> >> credentials,
>> >> >> >> pass a location to the flag where credentials for the registry
>> >> >> >> can be
>> >> >> found
>> >> >> >> (in JSON)? The frameworks can set credentials (maybe registry
>> >> >> >> name or
>> >> >> URL
>> >> >> >> to the registry), and the credentials can be learnt from the JSON
>> >> >> config.
>> >> >> >>
>> >> >> >
>> >> >> > What if we need credentials for multiple-registries? Have a JSON
>> >> >> > with
>> >> one
>> >> >> > credential per registry I guess? But if possible, I would love to
>> >> solve
>> >> >> > this more generally as possible; as Gilbert mentioned, this is not
>> >> >> > a
>> >> >> > problem just for Docker images but any URIs that need AuthN.
>> >> >> >
>> >> >> Yeah I was thinking having the JSON as a dictionary with keys being
>> >> >> the
>> >> >> registry URI (appc/docker) and the values being credentials (which
>> >> >> will
>> >> be
>> >> >> a dictionary as well I guess).
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Avinash Sridharan, Mesosphere
>> >> >> +1 (323) 702 5245
>> >> >>
>> >>
>> >>
>> >>
>> >> --
>> >> ~Kevin
>> >>
>>
>>
>>
>> --
>> ~Kevin
>
>
--
~Kevin
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Gilbert Song <gi...@mesosphere.io>.
@Kevin, thanks for writing it down in detail. It sounds good that a more
concrete
schema is designed to generally solve similar auth problem.
Just have two potential issues inlined below:
On Tue, Mar 15, 2016 at 5:39 PM, Kevin Klues <kl...@gmail.com> wrote:
> Yeah, option 2.
>
> I was trying to expand on Avinash's suggestion and make it a bit more
> concrete in terms of what was being proposed. Needing to reload the
> agent just to update the list of credentials it accepts seems
> undesirable though.
>
> Maybe we could have a way to start the agent with a default config (by
> iterating on the schema from my previous email), but allow newly
> launched frameworks to somehow update the config on the fly through a
>
Will it be too expensive to update all agents every time a new framework
joins (handling consensus problem as well)?
> file in their sandbox that follows the same schema.
>
Does that mean the file in sandbox should be exposed to each other?
> On Tue, Mar 15, 2016 at 5:25 PM, Jie Yu <yu...@gmail.com> wrote:
> > Kevin, are you suggesting option 2 and having a config file like the
> above?
> >
> > I think another downside of a per-agent config is that it's hard to
> > maintain this. What if a new framework joins and has a new credential for
> > the docker images. Do we need to restart the agent to reload the config?
> >
> > - Jie
> >
> > On Tue, Mar 15, 2016 at 1:25 PM, Kevin Klues <kl...@gmail.com> wrote:
> >
> >> Can we be a bit more concrete here and try to build up a schema for
> this.
> >> Maybe something like:
> >>
> >> {
> >> [
> >> {
> >> "service" : "docker",
> >> "registries" :
> >> [
> >> "uri" : "<uri>",
> >> "default_credentials" :
> >> {
> >> "type" : "<type>",
> >> "credential" :
> >> {
> >> // Custom based on type...
> >> }
> >> },
> >> "image_credentials" :
> >> [
> >> {
> >> "image_name" : "<image_name>",
> >> "type" : "<type>",
> >> "credential" :
> >> {
> >> // Custom based on type...
> >> },
> >> },
> >> ...
> >> ],
> >> ...
> >> ]
> >> ...
> >> },
> >> ...
> >> ]
> >> }
> >>
> >>
> >> On Tue, Mar 15, 2016 at 12:57 PM, Jie Yu <yu...@gmail.com> wrote:
> >> >>
> >> >> Yeah I was thinking having the JSON as a dictionary with keys being
> the
> >> >> registry URI (appc/docker) and the values being credentials (which
> will
> >> be
> >> >> a dictionary as well I guess).
> >> >
> >> >
> >> > Using registry URI as the key is problematic. Think about the public
> >> docker
> >> > hub. Different frameworks might want to use different credentials to
> >> access
> >> > their docker images.
> >> >
> >> > - Jie
> >> >
> >> > On Tue, Mar 15, 2016 at 11:52 AM, Avinash Sridharan <
> >> avinash@mesosphere.io
> >> >
> >> > wrote:
> >> >
> >> >> On Tue, Mar 15, 2016 at 11:43 AM, Vinod Kone <vi...@apache.org>
> >> wrote:
> >> >>
> >> >> > moved core@ to *bcc*
> >> >> >
> >> >> > On Tue, Mar 15, 2016 at 11:18 AM, Avinash Sridharan <
> >> >> avinash@mesosphere.io
> >> >> > > wrote:
> >> >> >
> >> >> >> Why not follow option 2, but instead of passing the agent
> >> credentials,
> >> >> >> pass a location to the flag where credentials for the registry
> can be
> >> >> found
> >> >> >> (in JSON)? The frameworks can set credentials (maybe registry
> name or
> >> >> URL
> >> >> >> to the registry), and the credentials can be learnt from the JSON
> >> >> config.
> >> >> >>
> >> >> >
> >> >> > What if we need credentials for multiple-registries? Have a JSON
> with
> >> one
> >> >> > credential per registry I guess? But if possible, I would love to
> >> solve
> >> >> > this more generally as possible; as Gilbert mentioned, this is not
> a
> >> >> > problem just for Docker images but any URIs that need AuthN.
> >> >> >
> >> >> Yeah I was thinking having the JSON as a dictionary with keys being
> the
> >> >> registry URI (appc/docker) and the values being credentials (which
> will
> >> be
> >> >> a dictionary as well I guess).
> >> >>
> >> >>
> >> >> --
> >> >> Avinash Sridharan, Mesosphere
> >> >> +1 (323) 702 5245
> >> >>
> >>
> >>
> >>
> >> --
> >> ~Kevin
> >>
>
>
>
> --
> ~Kevin
>
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Vinod Kone <vi...@apache.org>.
I actually think the way DockerContainerizer currently does it is
reasonable and allows each container to talk to a potentially different
registry using unique credentials. Given the multitude of AuthN schemes, it
is probably better to leave the problem of fetching CommandInfo.URIs that
need AuthN to fetcher modules.
On Tue, Mar 15, 2016 at 5:39 PM, Kevin Klues <kl...@gmail.com> wrote:
> Yeah, option 2.
>
> I was trying to expand on Avinash's suggestion and make it a bit more
> concrete in terms of what was being proposed. Needing to reload the
> agent just to update the list of credentials it accepts seems
> undesirable though.
>
> Maybe we could have a way to start the agent with a default config (by
> iterating on the schema from my previous email), but allow newly
> launched frameworks to somehow update the config on the fly through a
> file in their sandbox that follows the same schema.
>
> On Tue, Mar 15, 2016 at 5:25 PM, Jie Yu <yu...@gmail.com> wrote:
> > Kevin, are you suggesting option 2 and having a config file like the
> above?
> >
> > I think another downside of a per-agent config is that it's hard to
> > maintain this. What if a new framework joins and has a new credential for
> > the docker images. Do we need to restart the agent to reload the config?
> >
> > - Jie
> >
> > On Tue, Mar 15, 2016 at 1:25 PM, Kevin Klues <kl...@gmail.com> wrote:
> >
> >> Can we be a bit more concrete here and try to build up a schema for
> this.
> >> Maybe something like:
> >>
> >> {
> >> [
> >> {
> >> "service" : "docker",
> >> "registries" :
> >> [
> >> "uri" : "<uri>",
> >> "default_credentials" :
> >> {
> >> "type" : "<type>",
> >> "credential" :
> >> {
> >> // Custom based on type...
> >> }
> >> },
> >> "image_credentials" :
> >> [
> >> {
> >> "image_name" : "<image_name>",
> >> "type" : "<type>",
> >> "credential" :
> >> {
> >> // Custom based on type...
> >> },
> >> },
> >> ...
> >> ],
> >> ...
> >> ]
> >> ...
> >> },
> >> ...
> >> ]
> >> }
> >>
> >>
> >> On Tue, Mar 15, 2016 at 12:57 PM, Jie Yu <yu...@gmail.com> wrote:
> >> >>
> >> >> Yeah I was thinking having the JSON as a dictionary with keys being
> the
> >> >> registry URI (appc/docker) and the values being credentials (which
> will
> >> be
> >> >> a dictionary as well I guess).
> >> >
> >> >
> >> > Using registry URI as the key is problematic. Think about the public
> >> docker
> >> > hub. Different frameworks might want to use different credentials to
> >> access
> >> > their docker images.
> >> >
> >> > - Jie
> >> >
> >> > On Tue, Mar 15, 2016 at 11:52 AM, Avinash Sridharan <
> >> avinash@mesosphere.io
> >> >
> >> > wrote:
> >> >
> >> >> On Tue, Mar 15, 2016 at 11:43 AM, Vinod Kone <vi...@apache.org>
> >> wrote:
> >> >>
> >> >> > moved core@ to *bcc*
> >> >> >
> >> >> > On Tue, Mar 15, 2016 at 11:18 AM, Avinash Sridharan <
> >> >> avinash@mesosphere.io
> >> >> > > wrote:
> >> >> >
> >> >> >> Why not follow option 2, but instead of passing the agent
> >> credentials,
> >> >> >> pass a location to the flag where credentials for the registry
> can be
> >> >> found
> >> >> >> (in JSON)? The frameworks can set credentials (maybe registry
> name or
> >> >> URL
> >> >> >> to the registry), and the credentials can be learnt from the JSON
> >> >> config.
> >> >> >>
> >> >> >
> >> >> > What if we need credentials for multiple-registries? Have a JSON
> with
> >> one
> >> >> > credential per registry I guess? But if possible, I would love to
> >> solve
> >> >> > this more generally as possible; as Gilbert mentioned, this is not
> a
> >> >> > problem just for Docker images but any URIs that need AuthN.
> >> >> >
> >> >> Yeah I was thinking having the JSON as a dictionary with keys being
> the
> >> >> registry URI (appc/docker) and the values being credentials (which
> will
> >> be
> >> >> a dictionary as well I guess).
> >> >>
> >> >>
> >> >> --
> >> >> Avinash Sridharan, Mesosphere
> >> >> +1 (323) 702 5245
> >> >>
> >>
> >>
> >>
> >> --
> >> ~Kevin
> >>
>
>
>
> --
> ~Kevin
>
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Kevin Klues <kl...@gmail.com>.
Yeah, option 2.
I was trying to expand on Avinash's suggestion and make it a bit more
concrete in terms of what was being proposed. Needing to reload the
agent just to update the list of credentials it accepts seems
undesirable though.
Maybe we could have a way to start the agent with a default config (by
iterating on the schema from my previous email), but allow newly
launched frameworks to somehow update the config on the fly through a
file in their sandbox that follows the same schema.
On Tue, Mar 15, 2016 at 5:25 PM, Jie Yu <yu...@gmail.com> wrote:
> Kevin, are you suggesting option 2 and having a config file like the above?
>
> I think another downside of a per-agent config is that it's hard to
> maintain this. What if a new framework joins and has a new credential for
> the docker images. Do we need to restart the agent to reload the config?
>
> - Jie
>
> On Tue, Mar 15, 2016 at 1:25 PM, Kevin Klues <kl...@gmail.com> wrote:
>
>> Can we be a bit more concrete here and try to build up a schema for this.
>> Maybe something like:
>>
>> {
>> [
>> {
>> "service" : "docker",
>> "registries" :
>> [
>> "uri" : "<uri>",
>> "default_credentials" :
>> {
>> "type" : "<type>",
>> "credential" :
>> {
>> // Custom based on type...
>> }
>> },
>> "image_credentials" :
>> [
>> {
>> "image_name" : "<image_name>",
>> "type" : "<type>",
>> "credential" :
>> {
>> // Custom based on type...
>> },
>> },
>> ...
>> ],
>> ...
>> ]
>> ...
>> },
>> ...
>> ]
>> }
>>
>>
>> On Tue, Mar 15, 2016 at 12:57 PM, Jie Yu <yu...@gmail.com> wrote:
>> >>
>> >> Yeah I was thinking having the JSON as a dictionary with keys being the
>> >> registry URI (appc/docker) and the values being credentials (which will
>> be
>> >> a dictionary as well I guess).
>> >
>> >
>> > Using registry URI as the key is problematic. Think about the public
>> docker
>> > hub. Different frameworks might want to use different credentials to
>> access
>> > their docker images.
>> >
>> > - Jie
>> >
>> > On Tue, Mar 15, 2016 at 11:52 AM, Avinash Sridharan <
>> avinash@mesosphere.io
>> >
>> > wrote:
>> >
>> >> On Tue, Mar 15, 2016 at 11:43 AM, Vinod Kone <vi...@apache.org>
>> wrote:
>> >>
>> >> > moved core@ to *bcc*
>> >> >
>> >> > On Tue, Mar 15, 2016 at 11:18 AM, Avinash Sridharan <
>> >> avinash@mesosphere.io
>> >> > > wrote:
>> >> >
>> >> >> Why not follow option 2, but instead of passing the agent
>> credentials,
>> >> >> pass a location to the flag where credentials for the registry can be
>> >> found
>> >> >> (in JSON)? The frameworks can set credentials (maybe registry name or
>> >> URL
>> >> >> to the registry), and the credentials can be learnt from the JSON
>> >> config.
>> >> >>
>> >> >
>> >> > What if we need credentials for multiple-registries? Have a JSON with
>> one
>> >> > credential per registry I guess? But if possible, I would love to
>> solve
>> >> > this more generally as possible; as Gilbert mentioned, this is not a
>> >> > problem just for Docker images but any URIs that need AuthN.
>> >> >
>> >> Yeah I was thinking having the JSON as a dictionary with keys being the
>> >> registry URI (appc/docker) and the values being credentials (which will
>> be
>> >> a dictionary as well I guess).
>> >>
>> >>
>> >> --
>> >> Avinash Sridharan, Mesosphere
>> >> +1 (323) 702 5245
>> >>
>>
>>
>>
>> --
>> ~Kevin
>>
--
~Kevin
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Jie Yu <yu...@gmail.com>.
Kevin, are you suggesting option 2 and having a config file like the above?
I think another downside of a per-agent config is that it's hard to
maintain this. What if a new framework joins and has a new credential for
the docker images. Do we need to restart the agent to reload the config?
- Jie
On Tue, Mar 15, 2016 at 1:25 PM, Kevin Klues <kl...@gmail.com> wrote:
> Can we be a bit more concrete here and try to build up a schema for this.
> Maybe something like:
>
> {
> [
> {
> "service" : "docker",
> "registries" :
> [
> "uri" : "<uri>",
> "default_credentials" :
> {
> "type" : "<type>",
> "credential" :
> {
> // Custom based on type...
> }
> },
> "image_credentials" :
> [
> {
> "image_name" : "<image_name>",
> "type" : "<type>",
> "credential" :
> {
> // Custom based on type...
> },
> },
> ...
> ],
> ...
> ]
> ...
> },
> ...
> ]
> }
>
>
> On Tue, Mar 15, 2016 at 12:57 PM, Jie Yu <yu...@gmail.com> wrote:
> >>
> >> Yeah I was thinking having the JSON as a dictionary with keys being the
> >> registry URI (appc/docker) and the values being credentials (which will
> be
> >> a dictionary as well I guess).
> >
> >
> > Using registry URI as the key is problematic. Think about the public
> docker
> > hub. Different frameworks might want to use different credentials to
> access
> > their docker images.
> >
> > - Jie
> >
> > On Tue, Mar 15, 2016 at 11:52 AM, Avinash Sridharan <
> avinash@mesosphere.io
> >
> > wrote:
> >
> >> On Tue, Mar 15, 2016 at 11:43 AM, Vinod Kone <vi...@apache.org>
> wrote:
> >>
> >> > moved core@ to *bcc*
> >> >
> >> > On Tue, Mar 15, 2016 at 11:18 AM, Avinash Sridharan <
> >> avinash@mesosphere.io
> >> > > wrote:
> >> >
> >> >> Why not follow option 2, but instead of passing the agent
> credentials,
> >> >> pass a location to the flag where credentials for the registry can be
> >> found
> >> >> (in JSON)? The frameworks can set credentials (maybe registry name or
> >> URL
> >> >> to the registry), and the credentials can be learnt from the JSON
> >> config.
> >> >>
> >> >
> >> > What if we need credentials for multiple-registries? Have a JSON with
> one
> >> > credential per registry I guess? But if possible, I would love to
> solve
> >> > this more generally as possible; as Gilbert mentioned, this is not a
> >> > problem just for Docker images but any URIs that need AuthN.
> >> >
> >> Yeah I was thinking having the JSON as a dictionary with keys being the
> >> registry URI (appc/docker) and the values being credentials (which will
> be
> >> a dictionary as well I guess).
> >>
> >>
> >> --
> >> Avinash Sridharan, Mesosphere
> >> +1 (323) 702 5245
> >>
>
>
>
> --
> ~Kevin
>
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Kevin Klues <kl...@gmail.com>.
Can we be a bit more concrete here and try to build up a schema for this.
Maybe something like:
{
[
{
"service" : "docker",
"registries" :
[
"uri" : "<uri>",
"default_credentials" :
{
"type" : "<type>",
"credential" :
{
// Custom based on type...
}
},
"image_credentials" :
[
{
"image_name" : "<image_name>",
"type" : "<type>",
"credential" :
{
// Custom based on type...
},
},
...
],
...
]
...
},
...
]
}
On Tue, Mar 15, 2016 at 12:57 PM, Jie Yu <yu...@gmail.com> wrote:
>>
>> Yeah I was thinking having the JSON as a dictionary with keys being the
>> registry URI (appc/docker) and the values being credentials (which will
be
>> a dictionary as well I guess).
>
>
> Using registry URI as the key is problematic. Think about the public
docker
> hub. Different frameworks might want to use different credentials to
access
> their docker images.
>
> - Jie
>
> On Tue, Mar 15, 2016 at 11:52 AM, Avinash Sridharan <avinash@mesosphere.io
>
> wrote:
>
>> On Tue, Mar 15, 2016 at 11:43 AM, Vinod Kone <vi...@apache.org>
wrote:
>>
>> > moved core@ to *bcc*
>> >
>> > On Tue, Mar 15, 2016 at 11:18 AM, Avinash Sridharan <
>> avinash@mesosphere.io
>> > > wrote:
>> >
>> >> Why not follow option 2, but instead of passing the agent credentials,
>> >> pass a location to the flag where credentials for the registry can be
>> found
>> >> (in JSON)? The frameworks can set credentials (maybe registry name or
>> URL
>> >> to the registry), and the credentials can be learnt from the JSON
>> config.
>> >>
>> >
>> > What if we need credentials for multiple-registries? Have a JSON with
one
>> > credential per registry I guess? But if possible, I would love to solve
>> > this more generally as possible; as Gilbert mentioned, this is not a
>> > problem just for Docker images but any URIs that need AuthN.
>> >
>> Yeah I was thinking having the JSON as a dictionary with keys being the
>> registry URI (appc/docker) and the values being credentials (which will
be
>> a dictionary as well I guess).
>>
>>
>> --
>> Avinash Sridharan, Mesosphere
>> +1 (323) 702 5245
>>
--
~Kevin
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Jie Yu <yu...@gmail.com>.
>
> Yeah I was thinking having the JSON as a dictionary with keys being the
> registry URI (appc/docker) and the values being credentials (which will be
> a dictionary as well I guess).
Using registry URI as the key is problematic. Think about the public docker
hub. Different frameworks might want to use different credentials to access
their docker images.
- Jie
On Tue, Mar 15, 2016 at 11:52 AM, Avinash Sridharan <av...@mesosphere.io>
wrote:
> On Tue, Mar 15, 2016 at 11:43 AM, Vinod Kone <vi...@apache.org> wrote:
>
> > moved core@ to *bcc*
> >
> > On Tue, Mar 15, 2016 at 11:18 AM, Avinash Sridharan <
> avinash@mesosphere.io
> > > wrote:
> >
> >> Why not follow option 2, but instead of passing the agent credentials,
> >> pass a location to the flag where credentials for the registry can be
> found
> >> (in JSON)? The frameworks can set credentials (maybe registry name or
> URL
> >> to the registry), and the credentials can be learnt from the JSON
> config.
> >>
> >
> > What if we need credentials for multiple-registries? Have a JSON with one
> > credential per registry I guess? But if possible, I would love to solve
> > this more generally as possible; as Gilbert mentioned, this is not a
> > problem just for Docker images but any URIs that need AuthN.
> >
> Yeah I was thinking having the JSON as a dictionary with keys being the
> registry URI (appc/docker) and the values being credentials (which will be
> a dictionary as well I guess).
>
>
> --
> Avinash Sridharan, Mesosphere
> +1 (323) 702 5245
>
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Avinash Sridharan <av...@mesosphere.io>.
On Tue, Mar 15, 2016 at 11:43 AM, Vinod Kone <vi...@apache.org> wrote:
> moved core@ to *bcc*
>
> On Tue, Mar 15, 2016 at 11:18 AM, Avinash Sridharan <avinash@mesosphere.io
> > wrote:
>
>> Why not follow option 2, but instead of passing the agent credentials,
>> pass a location to the flag where credentials for the registry can be found
>> (in JSON)? The frameworks can set credentials (maybe registry name or URL
>> to the registry), and the credentials can be learnt from the JSON config.
>>
>
> What if we need credentials for multiple-registries? Have a JSON with one
> credential per registry I guess? But if possible, I would love to solve
> this more generally as possible; as Gilbert mentioned, this is not a
> problem just for Docker images but any URIs that need AuthN.
>
Yeah I was thinking having the JSON as a dictionary with keys being the
registry URI (appc/docker) and the values being credentials (which will be
a dictionary as well I guess).
--
Avinash Sridharan, Mesosphere
+1 (323) 702 5245
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Vinod Kone <vi...@apache.org>.
moved core@ to *bcc*
On Tue, Mar 15, 2016 at 11:18 AM, Avinash Sridharan <av...@mesosphere.io>
wrote:
> Why not follow option 2, but instead of passing the agent credentials,
> pass a location to the flag where credentials for the registry can be found
> (in JSON)? The frameworks can set credentials (maybe registry name or URL
> to the registry), and the credentials can be learnt from the JSON config.
>
What if we need credentials for multiple-registries? Have a JSON with one
credential per registry I guess? But if possible, I would love to solve
this more generally as possible; as Gilbert mentioned, this is not a
problem just for Docker images but any URIs that need AuthN.
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Jojy Varghese <jo...@mesosphere.io>.
Have we considered provisioning SSL certs and keys as a separate step (isolation maybe)? This could mean mounting “.ssl” volume inside the container for example.
-Jojy
> On Mar 15, 2016, at 11:18 AM, Avinash Sridharan <av...@mesosphere.io> wrote:
>
> Why not follow option 2, but instead of passing the agent credentials, pass a location to the flag where credentials for the registry can be found (in JSON)? The frameworks can set credentials (maybe registry name or URL to the registry), and the credentials can be learnt from the JSON config.
>
> With option 1, won't we need to mutate the frameworks to make them more secure ?
>
> On Tue, Mar 15, 2016 at 10:48 AM, Gilbert Song <gilbert@mesosphere.io <ma...@mesosphere.io>> wrote:
> Hi folks,
>
> We want to raise a discussion here, seeking suggestions about passing credentials in a secure way. This relates to the JIRA MESOS-4938 <https://issues.apache.org/jira/browse/MESOS-4938>, supporting docker private registry authentication in unified containerizer. In fact, this problem is not limited to docker registry. For instance, how can we support CommandInfo.URIs that need credentials?
>
> For the docker registry problem, credentials have to be included when communicating with the docker auth server. We have two options here:
>
> Option 1: Passing credentials in protobuf Image::Docker.
> Pros: This means supporting per-container docker registry, which is robust because different registry can be reached by an agent, configurable by users.
> Cons: So SSL has to be enabled to encrypt the communication between master and slave to prevent credentials from being seen by others. We also need to make sure we don’t expose credentials in any endpoint.
>
> Option 2: Passing credentials as an agent flag.
> Pros: Not necessary to be SSL enabled.
> Cons: No per-container registry support (imagine a multi-tenant cluster).
>
> Some background: How does docker containerizer solve this issue?
> In docker containerizer, we ask the framework to specify a URI for their task/executor that points to the .dockercfg(now ~/.docker/config.json) which contains the user and password information. The .dockercfg will be saved in the sandbox by the fetcher. When we call docker pull, we set the $HOME env for the subprocess to point to the sandbox so that the docker client can pick up that .dockercfg when pulling images.
>
> Any comment/advice will be absolutely welcome!
>
> Thanks,
> Gilbert/Jie
>
>
>
> --
> Avinash Sridharan, Mesosphere
> +1 (323) 702 5245
Re: [DISCUSS] Fetching Docker Images Requiring User Credentials.
Posted by Avinash Sridharan <av...@mesosphere.io>.
Why not follow option 2, but instead of passing the agent credentials, pass
a location to the flag where credentials for the registry can be found (in
JSON)? The frameworks can set credentials (maybe registry name or URL to
the registry), and the credentials can be learnt from the JSON config.
With option 1, won't we need to mutate the frameworks to make them more
secure ?
On Tue, Mar 15, 2016 at 10:48 AM, Gilbert Song <gi...@mesosphere.io>
wrote:
> Hi folks,
>
> We want to raise a discussion here, seeking suggestions about passing
> credentials in a secure way. This relates to the JIRA MESOS-4938
> <https://issues.apache.org/jira/browse/MESOS-4938>, supporting docker
> private registry authentication in unified containerizer. In fact, this
> problem is not limited to docker registry. For instance, how can we support
> CommandInfo.URIs that need credentials?
>
> For the docker registry problem, credentials have to be included when
> communicating with the docker auth server. We have two options here:
>
> Option 1: Passing credentials in protobuf Image::Docker.
>
> Pros: This means supporting per-container docker registry, which is
> robust because different registry can be reached by an agent, configurable
> by users.
>
> Cons: So SSL has to be enabled to encrypt the communication between
> master and slave to prevent credentials from being seen by others. We also
> need to make sure we don’t expose credentials in any endpoint.
>
> Option 2: Passing credentials as an agent flag.
>
> Pros: Not necessary to be SSL enabled.
>
> Cons: No per-container registry support (imagine a multi-tenant cluster).
>
> Some background: How does docker containerizer solve this issue?
>
> In docker containerizer, we ask the framework to specify a URI for their
> task/executor that points to the .dockercfg(now ~/.docker/config.json)
> which contains the user and password information. The .dockercfg will be
> saved in the sandbox by the fetcher. When we call docker pull, we set the
> $HOME env for the subprocess to point to the sandbox so that the docker
> client can pick up that .dockercfg when pulling images.
>
> Any comment/advice will be absolutely welcome!
>
> Thanks,
> Gilbert/Jie
>
--
Avinash Sridharan, Mesosphere
+1 (323) 702 5245