You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Mike Tutkowski <mi...@solidfire.com> on 2013/12/22 21:25:07 UTC

Distributed locking mechanism?

Hi,

I was wondering how we control access to shared resources that are being
utilized by different management servers at the same time.

For example:

When a user attaches a volume (that's based on the SolidFire plug-in) to a
VM, my plug-in looks at the CloudStack account that is requesting the
operation. If this CloudStack account does not have a corresponding account
on the SolidFire SAN, I must create one (there is a 1:1 mapping between
CloudStack and SolidFire accounts).

How can I protect against a situation where my plug-in is running in
multiple management servers and performing this kind of operation at the
same time (in other words, I don't want both instances of the plug-in to
see no SolidFire account and then they each end up creating one, which
breaks the 1:1 mapping between a CloudStack account and a SolidFire
account)?

Thanks!

-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Re: Distributed locking mechanism?

Posted by Chip Childers <ch...@gmail.com>.
On Mon, Dec 23, 2013 at 1:44 AM, Mike Tutkowski
<mi...@solidfire.com> wrote:
> Yeah, at first I was thinking both simultaneous calls would succeed and two
> accounts would be created (this is what happens when two volumes with the
> same name are created because the volume ID is what really matters).
>
> However, with accounts, the first will succeed and the others fail, which
> can be used to control the flow in this situation.

I'd be careful about assuming that won't change in the future, right?
Best to use the ACS locking mechanism than rely on something you can't
control in the remote device.

>
>
> On Sun, Dec 22, 2013 at 9:03 PM, Koushik Das <ko...@citrix.com> wrote:
>
>> What is the account creation logic on Solidfire SAN if you try to create
>> the same simultaneously? I am assuming there should be some unique
>> constraint like account name/id etc. Try to see if this can be utilised so
>> that one of the account creation call fails with appropriate
>> error/exception.
>>
>> -Koushik
>>
>> On 23-Dec-2013, at 1:55 AM, Mike Tutkowski <mi...@solidfire.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I was wondering how we control access to shared resources that are being
>> > utilized by different management servers at the same time.
>> >
>> > For example:
>> >
>> > When a user attaches a volume (that's based on the SolidFire plug-in) to
>> a
>> > VM, my plug-in looks at the CloudStack account that is requesting the
>> > operation. If this CloudStack account does not have a corresponding
>> account
>> > on the SolidFire SAN, I must create one (there is a 1:1 mapping between
>> > CloudStack and SolidFire accounts).
>> >
>> > How can I protect against a situation where my plug-in is running in
>> > multiple management servers and performing this kind of operation at the
>> > same time (in other words, I don't want both instances of the plug-in to
>> > see no SolidFire account and then they each end up creating one, which
>> > breaks the 1:1 mapping between a CloudStack account and a SolidFire
>> > account)?
>> >
>> > Thanks!
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > *™*
>>
>>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*

Re: Distributed locking mechanism?

Posted by Mike Tutkowski <mi...@solidfire.com>.
Yeah, at first I was thinking both simultaneous calls would succeed and two
accounts would be created (this is what happens when two volumes with the
same name are created because the volume ID is what really matters).

However, with accounts, the first will succeed and the others fail, which
can be used to control the flow in this situation.


On Sun, Dec 22, 2013 at 9:03 PM, Koushik Das <ko...@citrix.com> wrote:

> What is the account creation logic on Solidfire SAN if you try to create
> the same simultaneously? I am assuming there should be some unique
> constraint like account name/id etc. Try to see if this can be utilised so
> that one of the account creation call fails with appropriate
> error/exception.
>
> -Koushik
>
> On 23-Dec-2013, at 1:55 AM, Mike Tutkowski <mi...@solidfire.com>
> wrote:
>
> > Hi,
> >
> > I was wondering how we control access to shared resources that are being
> > utilized by different management servers at the same time.
> >
> > For example:
> >
> > When a user attaches a volume (that's based on the SolidFire plug-in) to
> a
> > VM, my plug-in looks at the CloudStack account that is requesting the
> > operation. If this CloudStack account does not have a corresponding
> account
> > on the SolidFire SAN, I must create one (there is a 1:1 mapping between
> > CloudStack and SolidFire accounts).
> >
> > How can I protect against a situation where my plug-in is running in
> > multiple management servers and performing this kind of operation at the
> > same time (in other words, I don't want both instances of the plug-in to
> > see no SolidFire account and then they each end up creating one, which
> > breaks the 1:1 mapping between a CloudStack account and a SolidFire
> > account)?
> >
> > Thanks!
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Re: Distributed locking mechanism?

Posted by Koushik Das <ko...@citrix.com>.
What is the account creation logic on Solidfire SAN if you try to create the same simultaneously? I am assuming there should be some unique constraint like account name/id etc. Try to see if this can be utilised so that one of the account creation call fails with appropriate error/exception.

-Koushik 

On 23-Dec-2013, at 1:55 AM, Mike Tutkowski <mi...@solidfire.com> wrote:

> Hi,
> 
> I was wondering how we control access to shared resources that are being
> utilized by different management servers at the same time.
> 
> For example:
> 
> When a user attaches a volume (that's based on the SolidFire plug-in) to a
> VM, my plug-in looks at the CloudStack account that is requesting the
> operation. If this CloudStack account does not have a corresponding account
> on the SolidFire SAN, I must create one (there is a 1:1 mapping between
> CloudStack and SolidFire accounts).
> 
> How can I protect against a situation where my plug-in is running in
> multiple management servers and performing this kind of operation at the
> same time (in other words, I don't want both instances of the plug-in to
> see no SolidFire account and then they each end up creating one, which
> breaks the 1:1 mapping between a CloudStack account and a SolidFire
> account)?
> 
> Thanks!
> 
> -- 
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*


Re: Distributed locking mechanism?

Posted by Mike Tutkowski <mi...@solidfire.com>.
Great, thanks, everyone.

Although I can get around this particular issue without the "InLockTable"
this time, "InLockTable" is really useful to know as I can see it coming
into play with other actions I may need to take.


On Sun, Dec 22, 2013 at 7:04 PM, Nitin Mehta <Ni...@citrix.com> wrote:

> Mike - I think your use case can be answered by Alex's wiki [1] - handling
> locking section.
> You can look for where all the methods provided in Generic Dao (that end
> in "InLockTable") is referred to get a better idea.
>
> [1]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Data+Access+Layer
>
> Thanks,
> -Nitin
>
> On 22/12/13 4:24 PM, "Mike Tutkowski" <mi...@solidfire.com>
> wrote:
>
> >Yeah, if it were just a single management server, I could use
> >"synchronized", but since it's not I was curious how we handle this
> >situation.
> >
> >Perhaps your recommendation in regards to using a database table is the
> >way
> >to go here. Do you know of any code in CloudStack that performs this kind
> >of action that I could use as a template?
> >
> >I actually looked into this SolidFire API more and - in this case -
> >account
> >name must be unique in the SAN, so it's a non-issue (I thought like
> >volumes
> >that account names could be reused and you would identify the one you want
> >based on its unique ID only). Technically it's not really a non-issue,
> >though. For example, two threads could try to create the same account at
> >the same time, but one would get an exception (duplicate account name).
> >That's OK, though. I could re-write the logic to catch the exception and
> >look for the account again (if the account is there, you're OK; else,
> >there
> >really is a problem).
> >
> >CloudStack doesn't need to know about SolidFire accounts, though (just my
> >plug-in does). SolidFire has this kind of division in our SAN to support
> >APIs that query for statistics based on a tenant's particular account.
> >
> >
> >On Sun, Dec 22, 2013 at 2:40 PM, Marcus Sorensen
> ><sh...@gmail.com>wrote:
> >
> >> As far as I'm aware, most of this sort of thing is managed by the state
> >> machines on the resources themselves, combined with the transactional
> >> database to move between states it keeps the management servers in sync.
> >>
> >> I'm not sure the problem is specific to having multiple mgmt servers.
> >>You'd
> >> have the same race with a single mgmt server, but having one makes it
> >> easier to solve (e.g. via synchronize block).
> >>
> >> There are probably many different ways to tackle this. My immediate
> >>thought
> >> is that cloudstack needs to know about your external accounts if its to
> >>be
> >> expected to handle them. But I imagine you could also use any old
> >>database
> >> transaction to do what you want, maybe have your own accounts table you
> >>can
> >> lock.
> >>
> >> Another option is simply to make the action idempotent. Why does
> >>creating
> >> the same account twice result in two accounts? Can that be fixed?
> >> On Dec 22, 2013 1:25 PM, "Mike Tutkowski" <mike.tutkowski@solidfire.com
> >
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > I was wondering how we control access to shared resources that are
> >>being
> >> > utilized by different management servers at the same time.
> >> >
> >> > For example:
> >> >
> >> > When a user attaches a volume (that's based on the SolidFire plug-in)
> >>to
> >> a
> >> > VM, my plug-in looks at the CloudStack account that is requesting the
> >> > operation. If this CloudStack account does not have a corresponding
> >> account
> >> > on the SolidFire SAN, I must create one (there is a 1:1 mapping
> >>between
> >> > CloudStack and SolidFire accounts).
> >> >
> >> > How can I protect against a situation where my plug-in is running in
> >> > multiple management servers and performing this kind of operation at
> >>the
> >> > same time (in other words, I don't want both instances of the plug-in
> >>to
> >> > see no SolidFire account and then they each end up creating one, which
> >> > breaks the 1:1 mapping between a CloudStack account and a SolidFire
> >> > account)?
> >> >
> >> > Thanks!
> >> >
> >> > --
> >> > *Mike Tutkowski*
> >> > *Senior CloudStack Developer, SolidFire Inc.*
> >> > e: mike.tutkowski@solidfire.com
> >> > o: 303.746.7302
> >> > Advancing the way the world uses the
> >> > cloud<http://solidfire.com/solution/overview/?video=play>
> >> > * *
> >> >
> >>
> >
> >
> >
> >--
> >*Mike Tutkowski*
> >*Senior CloudStack Developer, SolidFire Inc.*
> >e: mike.tutkowski@solidfire.com
> >o: 303.746.7302
> >Advancing the way the world uses the
> >cloud<http://solidfire.com/solution/overview/?video=play>
> >* *
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Re: Distributed locking mechanism?

Posted by John Griffith <jo...@solidfire.com>.
I think just catching/handling the exception here will give you exactly
what you want.  It doesn't eliminate the race but it mitigates any negative
impacts.  Maybe change your logic to first "get_account", then based on
results you can do the create or not
On Dec 22, 2013 5:25 PM, "Mike Tutkowski" <mi...@solidfire.com>
wrote:

> Yeah, if it were just a single management server, I could use
> "synchronized", but since it's not I was curious how we handle this
> situation.
>
> Perhaps your recommendation in regards to using a database table is the way
> to go here. Do you know of any code in CloudStack that performs this kind
> of action that I could use as a template?
>
> I actually looked into this SolidFire API more and - in this case - account
> name must be unique in the SAN, so it's a non-issue (I thought like volumes
> that account names could be reused and you would identify the one you want
> based on its unique ID only). Technically it's not really a non-issue,
> though. For example, two threads could try to create the same account at
> the same time, but one would get an exception (duplicate account name).
> That's OK, though. I could re-write the logic to catch the exception and
> look for the account again (if the account is there, you're OK; else, there
> really is a problem).
>
> CloudStack doesn't need to know about SolidFire accounts, though (just my
> plug-in does). SolidFire has this kind of division in our SAN to support
> APIs that query for statistics based on a tenant's particular account.
>
>
> On Sun, Dec 22, 2013 at 2:40 PM, Marcus Sorensen <shadowsor@gmail.com
> >wrote:
>
> > As far as I'm aware, most of this sort of thing is managed by the state
> > machines on the resources themselves, combined with the transactional
> > database to move between states it keeps the management servers in sync.
> >
> > I'm not sure the problem is specific to having multiple mgmt servers.
> You'd
> > have the same race with a single mgmt server, but having one makes it
> > easier to solve (e.g. via synchronize block).
> >
> > There are probably many different ways to tackle this. My immediate
> thought
> > is that cloudstack needs to know about your external accounts if its to
> be
> > expected to handle them. But I imagine you could also use any old
> database
> > transaction to do what you want, maybe have your own accounts table you
> can
> > lock.
> >
> > Another option is simply to make the action idempotent. Why does creating
> > the same account twice result in two accounts? Can that be fixed?
> > On Dec 22, 2013 1:25 PM, "Mike Tutkowski" <mi...@solidfire.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I was wondering how we control access to shared resources that are
> being
> > > utilized by different management servers at the same time.
> > >
> > > For example:
> > >
> > > When a user attaches a volume (that's based on the SolidFire plug-in)
> to
> > a
> > > VM, my plug-in looks at the CloudStack account that is requesting the
> > > operation. If this CloudStack account does not have a corresponding
> > account
> > > on the SolidFire SAN, I must create one (there is a 1:1 mapping between
> > > CloudStack and SolidFire accounts).
> > >
> > > How can I protect against a situation where my plug-in is running in
> > > multiple management servers and performing this kind of operation at
> the
> > > same time (in other words, I don't want both instances of the plug-in
> to
> > > see no SolidFire account and then they each end up creating one, which
> > > breaks the 1:1 mapping between a CloudStack account and a SolidFire
> > > account)?
> > >
> > > Thanks!
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkowski@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > *™*
> > >
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>

Re: Distributed locking mechanism?

Posted by Nitin Mehta <Ni...@citrix.com>.
Mike - I think your use case can be answered by Alex's wiki [1] - handling
locking section.		
You can look for where all the methods provided in Generic Dao (that end
in "InLockTable") is referred to get a better idea.

[1] 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Data+Access+Layer

Thanks,
-Nitin

On 22/12/13 4:24 PM, "Mike Tutkowski" <mi...@solidfire.com> wrote:

>Yeah, if it were just a single management server, I could use
>"synchronized", but since it's not I was curious how we handle this
>situation.
>
>Perhaps your recommendation in regards to using a database table is the
>way
>to go here. Do you know of any code in CloudStack that performs this kind
>of action that I could use as a template?
>
>I actually looked into this SolidFire API more and - in this case -
>account
>name must be unique in the SAN, so it's a non-issue (I thought like
>volumes
>that account names could be reused and you would identify the one you want
>based on its unique ID only). Technically it's not really a non-issue,
>though. For example, two threads could try to create the same account at
>the same time, but one would get an exception (duplicate account name).
>That's OK, though. I could re-write the logic to catch the exception and
>look for the account again (if the account is there, you're OK; else,
>there
>really is a problem).
>
>CloudStack doesn't need to know about SolidFire accounts, though (just my
>plug-in does). SolidFire has this kind of division in our SAN to support
>APIs that query for statistics based on a tenant's particular account.
>
>
>On Sun, Dec 22, 2013 at 2:40 PM, Marcus Sorensen
><sh...@gmail.com>wrote:
>
>> As far as I'm aware, most of this sort of thing is managed by the state
>> machines on the resources themselves, combined with the transactional
>> database to move between states it keeps the management servers in sync.
>>
>> I'm not sure the problem is specific to having multiple mgmt servers.
>>You'd
>> have the same race with a single mgmt server, but having one makes it
>> easier to solve (e.g. via synchronize block).
>>
>> There are probably many different ways to tackle this. My immediate
>>thought
>> is that cloudstack needs to know about your external accounts if its to
>>be
>> expected to handle them. But I imagine you could also use any old
>>database
>> transaction to do what you want, maybe have your own accounts table you
>>can
>> lock.
>>
>> Another option is simply to make the action idempotent. Why does
>>creating
>> the same account twice result in two accounts? Can that be fixed?
>> On Dec 22, 2013 1:25 PM, "Mike Tutkowski" <mi...@solidfire.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I was wondering how we control access to shared resources that are
>>being
>> > utilized by different management servers at the same time.
>> >
>> > For example:
>> >
>> > When a user attaches a volume (that's based on the SolidFire plug-in)
>>to
>> a
>> > VM, my plug-in looks at the CloudStack account that is requesting the
>> > operation. If this CloudStack account does not have a corresponding
>> account
>> > on the SolidFire SAN, I must create one (there is a 1:1 mapping
>>between
>> > CloudStack and SolidFire accounts).
>> >
>> > How can I protect against a situation where my plug-in is running in
>> > multiple management servers and performing this kind of operation at
>>the
>> > same time (in other words, I don't want both instances of the plug-in
>>to
>> > see no SolidFire account and then they each end up creating one, which
>> > breaks the 1:1 mapping between a CloudStack account and a SolidFire
>> > account)?
>> >
>> > Thanks!
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkowski@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud<http://solidfire.com/solution/overview/?video=play>
>> > **
>> >
>>
>
>
>
>-- 
>*Mike Tutkowski*
>*Senior CloudStack Developer, SolidFire Inc.*
>e: mike.tutkowski@solidfire.com
>o: 303.746.7302
>Advancing the way the world uses the
>cloud<http://solidfire.com/solution/overview/?video=play>
>**


Re: Distributed locking mechanism?

Posted by Mike Tutkowski <mi...@solidfire.com>.
Yeah, if it were just a single management server, I could use
"synchronized", but since it's not I was curious how we handle this
situation.

Perhaps your recommendation in regards to using a database table is the way
to go here. Do you know of any code in CloudStack that performs this kind
of action that I could use as a template?

I actually looked into this SolidFire API more and - in this case - account
name must be unique in the SAN, so it's a non-issue (I thought like volumes
that account names could be reused and you would identify the one you want
based on its unique ID only). Technically it's not really a non-issue,
though. For example, two threads could try to create the same account at
the same time, but one would get an exception (duplicate account name).
That's OK, though. I could re-write the logic to catch the exception and
look for the account again (if the account is there, you're OK; else, there
really is a problem).

CloudStack doesn't need to know about SolidFire accounts, though (just my
plug-in does). SolidFire has this kind of division in our SAN to support
APIs that query for statistics based on a tenant's particular account.


On Sun, Dec 22, 2013 at 2:40 PM, Marcus Sorensen <sh...@gmail.com>wrote:

> As far as I'm aware, most of this sort of thing is managed by the state
> machines on the resources themselves, combined with the transactional
> database to move between states it keeps the management servers in sync.
>
> I'm not sure the problem is specific to having multiple mgmt servers. You'd
> have the same race with a single mgmt server, but having one makes it
> easier to solve (e.g. via synchronize block).
>
> There are probably many different ways to tackle this. My immediate thought
> is that cloudstack needs to know about your external accounts if its to be
> expected to handle them. But I imagine you could also use any old database
> transaction to do what you want, maybe have your own accounts table you can
> lock.
>
> Another option is simply to make the action idempotent. Why does creating
> the same account twice result in two accounts? Can that be fixed?
> On Dec 22, 2013 1:25 PM, "Mike Tutkowski" <mi...@solidfire.com>
> wrote:
>
> > Hi,
> >
> > I was wondering how we control access to shared resources that are being
> > utilized by different management servers at the same time.
> >
> > For example:
> >
> > When a user attaches a volume (that's based on the SolidFire plug-in) to
> a
> > VM, my plug-in looks at the CloudStack account that is requesting the
> > operation. If this CloudStack account does not have a corresponding
> account
> > on the SolidFire SAN, I must create one (there is a 1:1 mapping between
> > CloudStack and SolidFire accounts).
> >
> > How can I protect against a situation where my plug-in is running in
> > multiple management servers and performing this kind of operation at the
> > same time (in other words, I don't want both instances of the plug-in to
> > see no SolidFire account and then they each end up creating one, which
> > breaks the 1:1 mapping between a CloudStack account and a SolidFire
> > account)?
> >
> > Thanks!
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkowski@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
> >
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*

Re: Distributed locking mechanism?

Posted by Marcus Sorensen <sh...@gmail.com>.
As far as I'm aware, most of this sort of thing is managed by the state
machines on the resources themselves, combined with the transactional
database to move between states it keeps the management servers in sync.

I'm not sure the problem is specific to having multiple mgmt servers. You'd
have the same race with a single mgmt server, but having one makes it
easier to solve (e.g. via synchronize block).

There are probably many different ways to tackle this. My immediate thought
is that cloudstack needs to know about your external accounts if its to be
expected to handle them. But I imagine you could also use any old database
transaction to do what you want, maybe have your own accounts table you can
lock.

Another option is simply to make the action idempotent. Why does creating
the same account twice result in two accounts? Can that be fixed?
On Dec 22, 2013 1:25 PM, "Mike Tutkowski" <mi...@solidfire.com>
wrote:

> Hi,
>
> I was wondering how we control access to shared resources that are being
> utilized by different management servers at the same time.
>
> For example:
>
> When a user attaches a volume (that's based on the SolidFire plug-in) to a
> VM, my plug-in looks at the CloudStack account that is requesting the
> operation. If this CloudStack account does not have a corresponding account
> on the SolidFire SAN, I must create one (there is a 1:1 mapping between
> CloudStack and SolidFire accounts).
>
> How can I protect against a situation where my plug-in is running in
> multiple management servers and performing this kind of operation at the
> same time (in other words, I don't want both instances of the plug-in to
> see no SolidFire account and then they each end up creating one, which
> breaks the 1:1 mapping between a CloudStack account and a SolidFire
> account)?
>
> Thanks!
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>