You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@deltacloud.apache.org by David Lutterkort <lu...@redhat.com> on 2010/11/12 02:21:24 UTC
Dynamic selection of driver
Hi,
Toby Crawley is really the one who brought this up - here's a summary of
a discussion we had a couple days ago.
Right now, the deltacloud server only ever uses one driver; that means
that if you want to talk to multiple clouds, you need to have one
deltacloud server running for each of them. Clearly, not very admin
friendly.
Toby actually experimented with this (wanna share your patch, Toby ?).
To me, the cleanest way to enable dynamic selection of the driver is to
select the driver through add'l HTTP headers. For example, by sending
X-Deltacloud-Driver: ec2
X-Deltacloud-Endpoint: https://eu-west-1.ec2.amazonaws.com/
you'd be using the eu-west region of EC2.
We would maintain the current behavior of the driver as the default when
these headers are absent; so if you start 'deltacloudd -i mock', clients
that don't set the Deltacloud headers will talk to the mock driver.
David
Re: Dynamic selection of driver
Posted by Mohammed Morsi <mm...@redhat.com>.
On 11/12/2010 04:32 AM, Michal Fojtik wrote:
> On 12/11/10 01:21 +0000, David Lutterkort wrote:
>> Hi,
>>
>> Toby Crawley is really the one who brought this up - here's a summary of
>> a discussion we had a couple days ago.
>>
>> Right now, the deltacloud server only ever uses one driver; that means
>> that if you want to talk to multiple clouds, you need to have one
>> deltacloud server running for each of them. Clearly, not very admin
>> friendly.
>>
>> Toby actually experimented with this (wanna share your patch, Toby ?).
>> To me, the cleanest way to enable dynamic selection of the driver is to
>> select the driver through add'l HTTP headers. For example, by sending
>>
>> X-Deltacloud-Driver: ec2
>> X-Deltacloud-Endpoint: https://eu-west-1.ec2.amazonaws.com/
>>
>> you'd be using the eu-west region of EC2.
>>
>> We would maintain the current behavior of the driver as the default when
>> these headers are absent; so if you start 'deltacloudd -i mock', clients
>> that don't set the Deltacloud headers will talk to the mock driver.
>
> I like this idea!. +1 from /me.
>
> -- Michal
>
Yes very good idea. +1 (never really liked the idea of having to launch
a bunch of core processes listening on different ports).
Also something else which I think would be appreciated by regular end
users is a combined client/server utility. Something which a user could
use to control a cloud provider without having to launch a core server.
They would checkout deltacloud core (or install it from the gem or rpm)
and just run deltacloud-embed (better name needed?) which launches a
server instance and communicates w/ it internally via the client
interface for a one-off operation. This would be the simplest / most
trivial deltacloud use case and I don't think implementing it would be
too bad either as we would just pull in the relevant bits from
deltacloudd and deltacloudc and write a very thin wrapper, though
performance might need to be optimized as the server currently takes a
few seconds to come up. Thoughts?
-Mo
Re: Dynamic selection of driver
Posted by Michal Fojtik <mf...@redhat.com>.
On 12/11/10 01:21 +0000, David Lutterkort wrote:
>Hi,
>
>Toby Crawley is really the one who brought this up - here's a summary of
>a discussion we had a couple days ago.
>
>Right now, the deltacloud server only ever uses one driver; that means
>that if you want to talk to multiple clouds, you need to have one
>deltacloud server running for each of them. Clearly, not very admin
>friendly.
>
>Toby actually experimented with this (wanna share your patch, Toby ?).
>To me, the cleanest way to enable dynamic selection of the driver is to
>select the driver through add'l HTTP headers. For example, by sending
>
> X-Deltacloud-Driver: ec2
> X-Deltacloud-Endpoint: https://eu-west-1.ec2.amazonaws.com/
>
>you'd be using the eu-west region of EC2.
>
>We would maintain the current behavior of the driver as the default when
>these headers are absent; so if you start 'deltacloudd -i mock', clients
>that don't set the Deltacloud headers will talk to the mock driver.
I like this idea!. +1 from /me.
-- Michal
--
--------------------------------------------------------
Michal Fojtik, mfojtik@redhat.com
Deltacloud API: http://deltacloud.org
--------------------------------------------------------
Re: Dynamic selection of driver
Posted by David Lutterkort <lu...@redhat.com>.
On Fri, 2010-11-12 at 09:29 -0500, Jim Jagielski wrote:
> On Nov 11, 2010, at 8:21 PM, David Lutterkort wrote:
>
> > Hi,
> >
> > Toby Crawley is really the one who brought this up - here's a summary of
> > a discussion we had a couple days ago.
> >
> > Right now, the deltacloud server only ever uses one driver; that means
> > that if you want to talk to multiple clouds, you need to have one
> > deltacloud server running for each of them. Clearly, not very admin
> > friendly.
> >
> > Toby actually experimented with this (wanna share your patch, Toby ?).
> > To me, the cleanest way to enable dynamic selection of the driver is to
> > select the driver through add'l HTTP headers. For example, by sending
> >
> > X-Deltacloud-Driver: ec2
> > X-Deltacloud-Endpoint: https://eu-west-1.ec2.amazonaws.com/
> >
> > you'd be using the eu-west region of EC2.
> >
> > We would maintain the current behavior of the driver as the default when
> > these headers are absent; so if you start 'deltacloudd -i mock', clients
> > that don't set the Deltacloud headers will talk to the mock driver.
> >
>
> Sounds good, but shouldn't there be some sort of auth mechanism
> to "verify" that an authorized entity set those headers?
I don't think that's necessary - since the Deltacloud server doesn't
store any credentials, the user also needs to include their credentials
with the request (as a HTTP basic auth header)
ASAICT, worst case, they have invalid credentials, and the only attack
they could launch is DOS either on the Deltacloud server (which they
could in any case) or on the backend cloud (which they also can do w/o
Deltacloud)
David
Re: Dynamic selection of driver
Posted by David Lutterkort <lu...@redhat.com>.
On Fri, 2010-11-12 at 12:07 -0500, Jim Jagielski wrote:
> On Nov 12, 2010, at 11:44 AM, Toby Crawley wrote:
>
> > On 11/12/2010 10:54 AM, Jim Jagielski wrote:
> >>
> >> On Nov 12, 2010, at 10:42 AM, Toby Crawley wrote:
> >>> Yes, ssl is not the only solution, but adds end to end security over the entire request, without adding complexity to the deltacloud code. I don't see a deltacloud service getting enough traffic to justify worrying about ssl overhead, and I would be more worried about someone extracting my cloud credentials, or munging the actual request itself than altering the driver or endpoint headers.
> >>>
> >>
> >> So it's either "trust nothing" or "trust everything"? :) Methinks that there's
> >> a middle ground somewhere in there *grin*
> >>
> >
> > Sure, there is a middle ground. My point is that if someone has the ability to munge my headers, then they have access to my cloud credentials, which is a much larger problem IMO.
> >
>
> Agreed. But that is hardly something totally unique in the HTTP
> space. My point is that if http headers will be used for control,
> then unless we have some sort of checks on them, then *only*
> ssl will be an option to enable some sort of trust value
> associated with that.
>
> Is that a long-term, viable condition?
Deltacloud doesn't trust the user - it doesn't care either way ;) But in
a serious production deployment, SSL should be used to protect the HTTP
basic auth header that the client uses to send its credentials for the
backend cloud.
David
Re: Dynamic selection of driver
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Nov 12, 2010, at 11:44 AM, Toby Crawley wrote:
> On 11/12/2010 10:54 AM, Jim Jagielski wrote:
>>
>> On Nov 12, 2010, at 10:42 AM, Toby Crawley wrote:
>>> Yes, ssl is not the only solution, but adds end to end security over the entire request, without adding complexity to the deltacloud code. I don't see a deltacloud service getting enough traffic to justify worrying about ssl overhead, and I would be more worried about someone extracting my cloud credentials, or munging the actual request itself than altering the driver or endpoint headers.
>>>
>>
>> So it's either "trust nothing" or "trust everything"? :) Methinks that there's
>> a middle ground somewhere in there *grin*
>>
>
> Sure, there is a middle ground. My point is that if someone has the ability to munge my headers, then they have access to my cloud credentials, which is a much larger problem IMO.
>
Agreed. But that is hardly something totally unique in the HTTP
space. My point is that if http headers will be used for control,
then unless we have some sort of checks on them, then *only*
ssl will be an option to enable some sort of trust value
associated with that.
Is that a long-term, viable condition?
Re: Dynamic selection of driver
Posted by Toby Crawley <tc...@redhat.com>.
On 11/12/2010 10:54 AM, Jim Jagielski wrote:
>
> On Nov 12, 2010, at 10:42 AM, Toby Crawley wrote:
>> Yes, ssl is not the only solution, but adds end to end security over the entire request, without adding complexity to the deltacloud code. I don't see a deltacloud service getting enough traffic to justify worrying about ssl overhead, and I would be more worried about someone extracting my cloud credentials, or munging the actual request itself than altering the driver or endpoint headers.
>>
>
> So it's either "trust nothing" or "trust everything"? :) Methinks that there's
> a middle ground somewhere in there *grin*
>
Sure, there is a middle ground. My point is that if someone has the ability to munge my headers, then they have access to my cloud
credentials, which is a much larger problem IMO.
Re: Dynamic selection of driver
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Nov 12, 2010, at 10:42 AM, Toby Crawley wrote:
> On 11/12/2010 10:26 AM, Jim Jagielski wrote:
>>
>> On Nov 12, 2010, at 10:04 AM, Toby Crawley wrote:
>>>
>>> If we suspect the source of the headers, we should suspect any data in the request. If an entity can munge headers, it can munge anything else in the request - the requests currently have no signing mechanism. If this type of security is a concern, the deltacloud server should be accessed via https. The client is based on RestClient, so should support https out of the box if deltacloud is running with a valid certificate. If using a self signed certificate, the client would probably need to be modified to not validate the server cert, or given the CA for the server cert so it can validate.
>>
>> That's not 100% true. Sure, one can munge headers; it's trivial. But what if
>> there is some simple md5 hash check, for example, related to the IP
>> address of the client and some shared secret. The client IP, since it's
>> NOT part of the http request, is not as easily munged...
>>
>> Any time you use http req headers as a control mechanism, you need
>> some sort of a&a mechanism to provide oversight. Even something as
>> simple a Digest auth (or some variant) provides *some* level of
>> protection w/o the full overhead of ssl.
>>
>> Of course, it goes w/o saying, that I'm offering to *add* this
>> capability ;)
>
> Yes, ssl is not the only solution, but adds end to end security over the entire request, without adding complexity to the deltacloud code. I don't see a deltacloud service getting enough traffic to justify worrying about ssl overhead, and I would be more worried about someone extracting my cloud credentials, or munging the actual request itself than altering the driver or endpoint headers.
>
So it's either "trust nothing" or "trust everything"? :) Methinks that there's
a middle ground somewhere in there *grin*
Re: Dynamic selection of driver
Posted by Toby Crawley <tc...@redhat.com>.
On 11/12/2010 10:26 AM, Jim Jagielski wrote:
>
> On Nov 12, 2010, at 10:04 AM, Toby Crawley wrote:
>>
>> If we suspect the source of the headers, we should suspect any data in the request. If an entity can munge headers, it can munge anything else in the request - the requests currently have no signing mechanism. If this type of security is a concern, the deltacloud server should be accessed via https. The client is based on RestClient, so should support https out of the box if deltacloud is running with a valid certificate. If using a self signed certificate, the client would probably need to be modified to not validate the server cert, or given the CA for the server cert so it can validate.
>
> That's not 100% true. Sure, one can munge headers; it's trivial. But what if
> there is some simple md5 hash check, for example, related to the IP
> address of the client and some shared secret. The client IP, since it's
> NOT part of the http request, is not as easily munged...
>
> Any time you use http req headers as a control mechanism, you need
> some sort of a&a mechanism to provide oversight. Even something as
> simple a Digest auth (or some variant) provides *some* level of
> protection w/o the full overhead of ssl.
>
> Of course, it goes w/o saying, that I'm offering to *add* this
> capability ;)
Yes, ssl is not the only solution, but adds end to end security over the entire request, without adding complexity to the deltacloud code. I
don't see a deltacloud service getting enough traffic to justify worrying about ssl overhead, and I would be more worried about someone
extracting my cloud credentials, or munging the actual request itself than altering the driver or endpoint headers.
Toby
Re: Dynamic selection of driver
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Nov 12, 2010, at 10:04 AM, Toby Crawley wrote:
> On 11/12/2010 09:29 AM, Jim Jagielski wrote:
>>
>> On Nov 11, 2010, at 8:21 PM, David Lutterkort wrote:
>>> Toby actually experimented with this (wanna share your patch, Toby ?).
>>> To me, the cleanest way to enable dynamic selection of the driver is to
>>> select the driver through add'l HTTP headers. For example, by sending
>>>
>>> X-Deltacloud-Driver: ec2
>>> X-Deltacloud-Endpoint: https://eu-west-1.ec2.amazonaws.com/
>>>
>>>
>>
>> Sounds good, but shouldn't there be some sort of auth mechanism
>> to "verify" that an authorized entity set those headers?
>>
>
> If we suspect the source of the headers, we should suspect any data in the request. If an entity can munge headers, it can munge anything else in the request - the requests currently have no signing mechanism. If this type of security is a concern, the deltacloud server should be accessed via https. The client is based on RestClient, so should support https out of the box if deltacloud is running with a valid certificate. If using a self signed certificate, the client would probably need to be modified to not validate the server cert, or given the CA for the server cert so it can validate.
That's not 100% true. Sure, one can munge headers; it's trivial. But what if
there is some simple md5 hash check, for example, related to the IP
address of the client and some shared secret. The client IP, since it's
NOT part of the http request, is not as easily munged...
Any time you use http req headers as a control mechanism, you need
some sort of a&a mechanism to provide oversight. Even something as
simple a Digest auth (or some variant) provides *some* level of
protection w/o the full overhead of ssl.
Of course, it goes w/o saying, that I'm offering to *add* this
capability ;)
Re: Dynamic selection of driver
Posted by Toby Crawley <tc...@redhat.com>.
On 11/12/2010 09:29 AM, Jim Jagielski wrote:
>
> On Nov 11, 2010, at 8:21 PM, David Lutterkort wrote:
>> Toby actually experimented with this (wanna share your patch, Toby ?).
>> To me, the cleanest way to enable dynamic selection of the driver is to
>> select the driver through add'l HTTP headers. For example, by sending
>>
>> X-Deltacloud-Driver: ec2
>> X-Deltacloud-Endpoint: https://eu-west-1.ec2.amazonaws.com/
>>
>>
>
> Sounds good, but shouldn't there be some sort of auth mechanism
> to "verify" that an authorized entity set those headers?
>
If we suspect the source of the headers, we should suspect any data in the request. If an entity can munge headers, it can munge anything
else in the request - the requests currently have no signing mechanism. If this type of security is a concern, the deltacloud server should
be accessed via https. The client is based on RestClient, so should support https out of the box if deltacloud is running with a valid
certificate. If using a self signed certificate, the client would probably need to be modified to not validate the server cert, or given the
CA for the server cert so it can validate.
Toby
Re: Dynamic selection of driver
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Nov 11, 2010, at 8:21 PM, David Lutterkort wrote:
> Hi,
>
> Toby Crawley is really the one who brought this up - here's a summary of
> a discussion we had a couple days ago.
>
> Right now, the deltacloud server only ever uses one driver; that means
> that if you want to talk to multiple clouds, you need to have one
> deltacloud server running for each of them. Clearly, not very admin
> friendly.
>
> Toby actually experimented with this (wanna share your patch, Toby ?).
> To me, the cleanest way to enable dynamic selection of the driver is to
> select the driver through add'l HTTP headers. For example, by sending
>
> X-Deltacloud-Driver: ec2
> X-Deltacloud-Endpoint: https://eu-west-1.ec2.amazonaws.com/
>
> you'd be using the eu-west region of EC2.
>
> We would maintain the current behavior of the driver as the default when
> these headers are absent; so if you start 'deltacloudd -i mock', clients
> that don't set the Deltacloud headers will talk to the mock driver.
>
Sounds good, but shouldn't there be some sort of auth mechanism
to "verify" that an authorized entity set those headers?
Re: Dynamic selection of driver
Posted by Chris Lalancette <cl...@redhat.com>.
On 11/12/10 - 01:21:24AM, David Lutterkort wrote:
> Hi,
>
> Toby Crawley is really the one who brought this up - here's a summary of
> a discussion we had a couple days ago.
>
> Right now, the deltacloud server only ever uses one driver; that means
> that if you want to talk to multiple clouds, you need to have one
> deltacloud server running for each of them. Clearly, not very admin
> friendly.
>
> Toby actually experimented with this (wanna share your patch, Toby ?).
> To me, the cleanest way to enable dynamic selection of the driver is to
> select the driver through add'l HTTP headers. For example, by sending
>
> X-Deltacloud-Driver: ec2
> X-Deltacloud-Endpoint: https://eu-west-1.ec2.amazonaws.com/
>
> you'd be using the eu-west region of EC2.
>
> We would maintain the current behavior of the driver as the default when
> these headers are absent; so if you start 'deltacloudd -i mock', clients
> that don't set the Deltacloud headers will talk to the mock driver.
Yes, excellent idea.
--
Chris Lalancette
Re: Dynamic selection of driver
Posted by Toby Crawley <tc...@redhat.com>.
On 11/11/2010 08:21 PM, David Lutterkort wrote:
> Hi,
>
> Toby Crawley is really the one who brought this up - here's a summary of
> a discussion we had a couple days ago.
>
> Right now, the deltacloud server only ever uses one driver; that means
> that if you want to talk to multiple clouds, you need to have one
> deltacloud server running for each of them. Clearly, not very admin
> friendly.
>
> Toby actually experimented with this (wanna share your patch, Toby ?).
> To me, the cleanest way to enable dynamic selection of the driver is to
> select the driver through add'l HTTP headers. For example, by sending
>
> X-Deltacloud-Driver: ec2
> X-Deltacloud-Endpoint: https://eu-west-1.ec2.amazonaws.com/
>
> you'd be using the eu-west region of EC2.
>
> We would maintain the current behavior of the driver as the default when
> these headers are absent; so if you start 'deltacloudd -i mock', clients
> that don't set the Deltacloud headers will talk to the mock driver.
>
> David
The 'patch' I have is really just a hack - it monkey patches client/deltacloud to send the ec2 endpoint as a header, and provides an
alternate rackup file for the server that inserts rack middleware to turn headers in to ENV vars. I had not yet added changes to override
the driver, but that would be fairly straightforward. I took this approach to see if it could be done without altering the -client and -core
gems. The code is at https://gist.github.com/664838. If we bake this functionality in to the client, it should probably baked in to the
server as well, instead of using the middleware approach.
Toby
Re: Dynamic selection of driver
Posted by David Lutterkort <lu...@redhat.com>.
On Fri, 2010-11-12 at 09:09 -0500, Benjamin Browning wrote:
> On Nov 11, 2010, at 8:21 PM, David Lutterkort wrote:
>
> > Right now, the deltacloud server only ever uses one driver; that means
> > that if you want to talk to multiple clouds, you need to have one
> > deltacloud server running for each of them. Clearly, not very admin
> > friendly.
> >
> > Toby actually experimented with this (wanna share your patch, Toby ?).
> > To me, the cleanest way to enable dynamic selection of the driver is to
> > select the driver through add'l HTTP headers. For example, by sending
> >
> > X-Deltacloud-Driver: ec2
> > X-Deltacloud-Endpoint: https://eu-west-1.ec2.amazonaws.com/
> >
> > you'd be using the eu-west region of EC2.
> >
> > We would maintain the current behavior of the driver as the default when
> > these headers are absent; so if you start 'deltacloudd -i mock', clients
> > that don't set the Deltacloud headers will talk to the mock driver.
> >
>
> +1 from me as well. Should each server advertise the drivers it
> supports, or is it reasonable to assume the user knows which drivers
> are supported for the server they're connected to?
We should probably advertise the list of available drivers. Doesn't feel
quite right to do it in the entrypoint XML, since that's already driver
specific; the other option would be to do it with headers, but that
seems a little obscure.
David
Re: Dynamic selection of driver
Posted by Benjamin Browning <bb...@redhat.com>.
On Nov 11, 2010, at 8:21 PM, David Lutterkort wrote:
> Right now, the deltacloud server only ever uses one driver; that means
> that if you want to talk to multiple clouds, you need to have one
> deltacloud server running for each of them. Clearly, not very admin
> friendly.
>
> Toby actually experimented with this (wanna share your patch, Toby ?).
> To me, the cleanest way to enable dynamic selection of the driver is to
> select the driver through add'l HTTP headers. For example, by sending
>
> X-Deltacloud-Driver: ec2
> X-Deltacloud-Endpoint: https://eu-west-1.ec2.amazonaws.com/
>
> you'd be using the eu-west region of EC2.
>
> We would maintain the current behavior of the driver as the default when
> these headers are absent; so if you start 'deltacloudd -i mock', clients
> that don't set the Deltacloud headers will talk to the mock driver.
>
+1 from me as well. Should each server advertise the drivers it supports, or is it reasonable to assume the user knows which drivers are supported for the server they're connected to?