You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Alena Prokharchyk <Al...@citrix.com> on 2014/02/01 01:26:34 UTC
[PROPOSAL] Introduce API returning you an answer from CloudStack
storage/host allocators whethere there is enough resources for vm deployment
Currently there is no way to know if there is enough resources for vm deployment, before actual deployVm call is made. The sequence is the following:
1) Deploy Vm is called
2) DB record is created for the Vm
3) Storage/Host allocators determine whethere there are enough resources for vm to be deployed, and return deploy destination to the caller stack.
4) If allocator returns valid deploy destination, VM gets actually created/started on the backend. If allocators don't return the destination, the DB record created on step 2) gets destroyed, and ResourceAllocation exception is thrown back to the API caller.
The API I'm going to introduce, would help you to determine whether CS physical resources - hosts, storages - can potentially accomodate vm deployment (considering template/service/diskOffering) at a given time, w/o actually calling the deploy vm. Some admins might find this call useful as they can always make this check before submitting the deployVm, so in case it returns NO, you can fail the deployment immediately, w/o calling deployVm. Also you can make this call to determine what is lacking for certain vm deployment, and expand your physical resources accordingly.
Please let me know if see any pitfalls in the proposal, as well if you see any other use cases that can be solved using this API.
Prachi, can you please point me to an existing method (or interface) defined in Allocators code serving this purpose?
Thanks,
-Alena.
Re: [PROPOSAL] Introduce API returning you an answer from CloudStack
storage/host allocators whethere there is enough resources for vm deployment
Posted by Yitao Jiang <wi...@gmail.com>.
Great idea.i supposed an api could return the remained resources, and then
deciding which host,both storage and compute, to deploy vm.
Koushik Das <ko...@citrix.com>于2014年2月3日星期一写道：
> As I understand this is a best-effort approach. Even if the proposed API
returns YES, deployment may still fail. I personally feel that it is better
to improve the deployVM API itself to return accurate error message rather
than having multiple API calls. If the deployvm call call fails, it will
clearly indicate reason for failure so appropriate action can be taken.
>
> -Koushik
>
> On 03-Feb-2014, at 12:35 PM, Rajani Karuturi <Ra...@citrix.com>
wrote:
>
>> I agree with Santhosh. Returning appropriate error codes might be enough.
>>
>> Thanks,
>> ~Rajani
>>
>>
>>
>> On 02-Feb-2014, at 11:17 pm, Santhosh Edukulla <
santhosh.edukulla@citrix.com> wrote:
>>
>>> Just a note:
>>>
>>> Instead of making two separate api calls one to check and then for
deploy, may be it can be part of deploy only and check these conditions for
resources\any other information and return appropriate codes? This check
can be called firsthand as part of deploy\other commands applicable and
return with codes before proceeding further with steps of deploy mentioned.
>>>
>>> Thanks!
>>> Santhosh
>>> ________________________________________
>>> From: Daan Hoogland [daan.hoogland@gmail.com]
>>> Sent: Saturday, February 01, 2014 11:24 AM
>>> To: dev
>>> Subject: Re: [PROPOSAL] Introduce API returning you an answer from
CloudStack storage/host allocators whethere there is enough resources for
vm deployment
>>>
>>> I like the idea. Would the api query the db or the bakends (i.e. hosts,
>>> storage, etc)?
>>>
>>> mobile bilingual spell checker used
>>> Op 1 feb. 2014 17:13 schreef "abhisek basu" <ab...@msn.com>:
>>>
>>>> Great idea, this would certainly reduce vm deployment failures.
>>>>
>>>> VR including RVR if needs to be created, should get included in this
check.
>>>>
>>>> Also, can we include ways to check if a VM can be launched in a certain
>>>> cluster? The idea is to allow users / admins to be able to launch vm
in a
>>>> certain cluster, but that might be the next functionality.
>>>>
>>>> Sent from my iPhone
>>>>
>>>>> On 1 Feb 2014, at 8:50 pm, "Ryan Lei" <ry...@cht.com.tw> wrote:
>>>>>
>>>>> I believe it's a nice idea. A lot of us have experienced the
>>>>> annoying InsufficientCapacityException in Deploying Virtual Machines,
but
>>>>> we can't tell exactly why just by reading the logs.
>>>>>
>>>>> If this new API could help us debug this VM deployment process to
>>>> determine
>>>>> exactly which resource is lacking, or probably other internal reasons
>>>> that
>>>>> cause this InsufficientCapacityException, it would be very helpful!
>>>>>
>>>>>
>>>>
-------------------------------------------------------------------------------------------
>>>>> Yu-Heng (Ryan) Lei, Associate Researcher
>>>>> Cloud Computing Dept, Chunghwa Telecom Labs
>>>>> ryanlei@cht.com.tw or ryanlei750328@gmail.com
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Feb 1, 2014 at 8:26 AM, Alena Prokharchyk <
>>>>> Alena.Prokharchyk@citrix.com> wrote:
>>>>>
>>>>>> Currently there is no way to know if there is enough resources for vm
>>>>>> deployment, before actual deployVm call is made. The sequence is the
>>>>>> following:
>>>>>>
>>>>>> 1) Deploy Vm is called
>>>>>> 2) DB record is created for the Vm
>>>>>> 3) Storage/Host allocators determine whethere there are enough
resources
>>>>>> for vm to be deployed, and return deploy destination to the caller
>>>> stack.
>>>>>> 4) If allocator returns valid deploy destination, VM gets actually
>>>>>> created/started on the backend. If allocators don't return the
>>>> destination,
>>>>>> the DB record created on step 2) gets destroyed, and
ResourceAllocation
>>>>>> exception is thrown back to the API caller.
>>>>
--
Thanks,
Yitao
Re: [PROPOSAL] Introduce API returning you an answer from
CloudStack storage/host allocators whethere there is enough resources for vm
deployment
Posted by Koushik Das <ko...@citrix.com>.
As I understand this is a best-effort approach. Even if the proposed API returns YES, deployment may still fail. I personally feel that it is better to improve the deployVM API itself to return accurate error message rather than having multiple API calls. If the deployvm call call fails, it will clearly indicate reason for failure so appropriate action can be taken.
-Koushik
On 03-Feb-2014, at 12:35 PM, Rajani Karuturi <Ra...@citrix.com> wrote:
> I agree with Santhosh. Returning appropriate error codes might be enough.
>
> Thanks,
> ~Rajani
>
>
>
> On 02-Feb-2014, at 11:17 pm, Santhosh Edukulla <sa...@citrix.com> wrote:
>
>> Just a note:
>>
>> Instead of making two separate api calls one to check and then for deploy, may be it can be part of deploy only and check these conditions for resources\any other information and return appropriate codes? This check can be called firsthand as part of deploy\other commands applicable and return with codes before proceeding further with steps of deploy mentioned.
>>
>> Thanks!
>> Santhosh
>> ________________________________________
>> From: Daan Hoogland [daan.hoogland@gmail.com]
>> Sent: Saturday, February 01, 2014 11:24 AM
>> To: dev
>> Subject: Re: [PROPOSAL] Introduce API returning you an answer from CloudStack storage/host allocators whethere there is enough resources for vm deployment
>>
>> I like the idea. Would the api query the db or the bakends (i.e. hosts,
>> storage, etc)?
>>
>> mobile bilingual spell checker used
>> Op 1 feb. 2014 17:13 schreef "abhisek basu" <ab...@msn.com>:
>>
>>> Great idea, this would certainly reduce vm deployment failures.
>>>
>>> VR including RVR if needs to be created, should get included in this check.
>>>
>>> Also, can we include ways to check if a VM can be launched in a certain
>>> cluster? The idea is to allow users / admins to be able to launch vm in a
>>> certain cluster, but that might be the next functionality.
>>>
>>> Sent from my iPhone
>>>
>>>> On 1 Feb 2014, at 8:50 pm, "Ryan Lei" <ry...@cht.com.tw> wrote:
>>>>
>>>> I believe it's a nice idea. A lot of us have experienced the
>>>> annoying InsufficientCapacityException in Deploying Virtual Machines, but
>>>> we can't tell exactly why just by reading the logs.
>>>>
>>>> If this new API could help us debug this VM deployment process to
>>> determine
>>>> exactly which resource is lacking, or probably other internal reasons
>>> that
>>>> cause this InsufficientCapacityException, it would be very helpful!
>>>>
>>>>
>>> -------------------------------------------------------------------------------------------
>>>> Yu-Heng (Ryan) Lei, Associate Researcher
>>>> Cloud Computing Dept, Chunghwa Telecom Labs
>>>> ryanlei@cht.com.tw or ryanlei750328@gmail.com
>>>>
>>>>
>>>>
>>>> On Sat, Feb 1, 2014 at 8:26 AM, Alena Prokharchyk <
>>>> Alena.Prokharchyk@citrix.com> wrote:
>>>>
>>>>> Currently there is no way to know if there is enough resources for vm
>>>>> deployment, before actual deployVm call is made. The sequence is the
>>>>> following:
>>>>>
>>>>> 1) Deploy Vm is called
>>>>> 2) DB record is created for the Vm
>>>>> 3) Storage/Host allocators determine whethere there are enough resources
>>>>> for vm to be deployed, and return deploy destination to the caller
>>> stack.
>>>>> 4) If allocator returns valid deploy destination, VM gets actually
>>>>> created/started on the backend. If allocators don't return the
>>> destination,
>>>>> the DB record created on step 2) gets destroyed, and ResourceAllocation
>>>>> exception is thrown back to the API caller.
>>>>>
>>>>> The API I'm going to introduce, would help you to determine whether CS
>>>>> physical resources - hosts, storages - can potentially accomodate vm
>>>>> deployment (considering template/service/diskOffering) at a given time,
>>> w/o
>>>>> actually calling the deploy vm. Some admins might find this call useful
>>> as
>>>>> they can always make this check before submitting the deployVm, so in
>>> case
>>>>> it returns NO, you can fail the deployment immediately, w/o calling
>>>>> deployVm. Also you can make this call to determine what is lacking for
>>>>> certain vm deployment, and expand your physical resources accordingly.
>>>>>
>>>>> Please let me know if see any pitfalls in the proposal, as well if you
>>> see
>>>>> any other use cases that can be solved using this API.
>>>>>
>>>>> Prachi, can you please point me to an existing method (or interface)
>>>>> defined in Allocators code serving this purpose?
>>>>>
>>>>> Thanks,
>>>>> -Alena.
>>>>>
>>>
>
Re: [PROPOSAL] Introduce API returning you an answer from
CloudStack storage/host allocators whethere there is enough resources for vm
deployment
Posted by Rajani Karuturi <Ra...@citrix.com>.
I agree with Santhosh. Returning appropriate error codes might be enough.
Thanks,
~Rajani
On 02-Feb-2014, at 11:17 pm, Santhosh Edukulla <sa...@citrix.com> wrote:
> Just a note:
>
> Instead of making two separate api calls one to check and then for deploy, may be it can be part of deploy only and check these conditions for resources\any other information and return appropriate codes? This check can be called firsthand as part of deploy\other commands applicable and return with codes before proceeding further with steps of deploy mentioned.
>
> Thanks!
> Santhosh
> ________________________________________
> From: Daan Hoogland [daan.hoogland@gmail.com]
> Sent: Saturday, February 01, 2014 11:24 AM
> To: dev
> Subject: Re: [PROPOSAL] Introduce API returning you an answer from CloudStack storage/host allocators whethere there is enough resources for vm deployment
>
> I like the idea. Would the api query the db or the bakends (i.e. hosts,
> storage, etc)?
>
> mobile bilingual spell checker used
> Op 1 feb. 2014 17:13 schreef "abhisek basu" <ab...@msn.com>:
>
>> Great idea, this would certainly reduce vm deployment failures.
>>
>> VR including RVR if needs to be created, should get included in this check.
>>
>> Also, can we include ways to check if a VM can be launched in a certain
>> cluster? The idea is to allow users / admins to be able to launch vm in a
>> certain cluster, but that might be the next functionality.
>>
>> Sent from my iPhone
>>
>>> On 1 Feb 2014, at 8:50 pm, "Ryan Lei" <ry...@cht.com.tw> wrote:
>>>
>>> I believe it's a nice idea. A lot of us have experienced the
>>> annoying InsufficientCapacityException in Deploying Virtual Machines, but
>>> we can't tell exactly why just by reading the logs.
>>>
>>> If this new API could help us debug this VM deployment process to
>> determine
>>> exactly which resource is lacking, or probably other internal reasons
>> that
>>> cause this InsufficientCapacityException, it would be very helpful!
>>>
>>>
>> -------------------------------------------------------------------------------------------
>>> Yu-Heng (Ryan) Lei, Associate Researcher
>>> Cloud Computing Dept, Chunghwa Telecom Labs
>>> ryanlei@cht.com.tw or ryanlei750328@gmail.com
>>>
>>>
>>>
>>> On Sat, Feb 1, 2014 at 8:26 AM, Alena Prokharchyk <
>>> Alena.Prokharchyk@citrix.com> wrote:
>>>
>>>> Currently there is no way to know if there is enough resources for vm
>>>> deployment, before actual deployVm call is made. The sequence is the
>>>> following:
>>>>
>>>> 1) Deploy Vm is called
>>>> 2) DB record is created for the Vm
>>>> 3) Storage/Host allocators determine whethere there are enough resources
>>>> for vm to be deployed, and return deploy destination to the caller
>> stack.
>>>> 4) If allocator returns valid deploy destination, VM gets actually
>>>> created/started on the backend. If allocators don't return the
>> destination,
>>>> the DB record created on step 2) gets destroyed, and ResourceAllocation
>>>> exception is thrown back to the API caller.
>>>>
>>>> The API I'm going to introduce, would help you to determine whether CS
>>>> physical resources - hosts, storages - can potentially accomodate vm
>>>> deployment (considering template/service/diskOffering) at a given time,
>> w/o
>>>> actually calling the deploy vm. Some admins might find this call useful
>> as
>>>> they can always make this check before submitting the deployVm, so in
>> case
>>>> it returns NO, you can fail the deployment immediately, w/o calling
>>>> deployVm. Also you can make this call to determine what is lacking for
>>>> certain vm deployment, and expand your physical resources accordingly.
>>>>
>>>> Please let me know if see any pitfalls in the proposal, as well if you
>> see
>>>> any other use cases that can be solved using this API.
>>>>
>>>> Prachi, can you please point me to an existing method (or interface)
>>>> defined in Allocators code serving this purpose?
>>>>
>>>> Thanks,
>>>> -Alena.
>>>>
>>
RE: [PROPOSAL] Introduce API returning you an answer from
CloudStack storage/host allocators whethere there is enough resources for vm
deployment
Posted by Santhosh Edukulla <sa...@citrix.com>.
Just a note:
Instead of making two separate api calls one to check and then for deploy, may be it can be part of deploy only and check these conditions for resources\any other information and return appropriate codes? This check can be called firsthand as part of deploy\other commands applicable and return with codes before proceeding further with steps of deploy mentioned.
Thanks!
Santhosh
________________________________________
From: Daan Hoogland [daan.hoogland@gmail.com]
Sent: Saturday, February 01, 2014 11:24 AM
To: dev
Subject: Re: [PROPOSAL] Introduce API returning you an answer from CloudStack storage/host allocators whethere there is enough resources for vm deployment
I like the idea. Would the api query the db or the bakends (i.e. hosts,
storage, etc)?
mobile bilingual spell checker used
Op 1 feb. 2014 17:13 schreef "abhisek basu" <ab...@msn.com>:
> Great idea, this would certainly reduce vm deployment failures.
>
> VR including RVR if needs to be created, should get included in this check.
>
> Also, can we include ways to check if a VM can be launched in a certain
> cluster? The idea is to allow users / admins to be able to launch vm in a
> certain cluster, but that might be the next functionality.
>
> Sent from my iPhone
>
> > On 1 Feb 2014, at 8:50 pm, "Ryan Lei" <ry...@cht.com.tw> wrote:
> >
> > I believe it's a nice idea. A lot of us have experienced the
> > annoying InsufficientCapacityException in Deploying Virtual Machines, but
> > we can't tell exactly why just by reading the logs.
> >
> > If this new API could help us debug this VM deployment process to
> determine
> > exactly which resource is lacking, or probably other internal reasons
> that
> > cause this InsufficientCapacityException, it would be very helpful!
> >
> >
> -------------------------------------------------------------------------------------------
> > Yu-Heng (Ryan) Lei, Associate Researcher
> > Cloud Computing Dept, Chunghwa Telecom Labs
> > ryanlei@cht.com.tw or ryanlei750328@gmail.com
> >
> >
> >
> > On Sat, Feb 1, 2014 at 8:26 AM, Alena Prokharchyk <
> > Alena.Prokharchyk@citrix.com> wrote:
> >
> >> Currently there is no way to know if there is enough resources for vm
> >> deployment, before actual deployVm call is made. The sequence is the
> >> following:
> >>
> >> 1) Deploy Vm is called
> >> 2) DB record is created for the Vm
> >> 3) Storage/Host allocators determine whethere there are enough resources
> >> for vm to be deployed, and return deploy destination to the caller
> stack.
> >> 4) If allocator returns valid deploy destination, VM gets actually
> >> created/started on the backend. If allocators don't return the
> destination,
> >> the DB record created on step 2) gets destroyed, and ResourceAllocation
> >> exception is thrown back to the API caller.
> >>
> >> The API I'm going to introduce, would help you to determine whether CS
> >> physical resources - hosts, storages - can potentially accomodate vm
> >> deployment (considering template/service/diskOffering) at a given time,
> w/o
> >> actually calling the deploy vm. Some admins might find this call useful
> as
> >> they can always make this check before submitting the deployVm, so in
> case
> >> it returns NO, you can fail the deployment immediately, w/o calling
> >> deployVm. Also you can make this call to determine what is lacking for
> >> certain vm deployment, and expand your physical resources accordingly.
> >>
> >> Please let me know if see any pitfalls in the proposal, as well if you
> see
> >> any other use cases that can be solved using this API.
> >>
> >> Prachi, can you please point me to an existing method (or interface)
> >> defined in Allocators code serving this purpose?
> >>
> >> Thanks,
> >> -Alena.
> >>
>
Re: [PROPOSAL] Introduce API returning you an answer from CloudStack
storage/host allocators whethere there is enough resources for vm deployment
Posted by Daan Hoogland <da...@gmail.com>.
I like the idea. Would the api query the db or the bakends (i.e. hosts,
storage, etc)?
mobile bilingual spell checker used
Op 1 feb. 2014 17:13 schreef "abhisek basu" <ab...@msn.com>:
> Great idea, this would certainly reduce vm deployment failures.
>
> VR including RVR if needs to be created, should get included in this check.
>
> Also, can we include ways to check if a VM can be launched in a certain
> cluster? The idea is to allow users / admins to be able to launch vm in a
> certain cluster, but that might be the next functionality.
>
> Sent from my iPhone
>
> > On 1 Feb 2014, at 8:50 pm, "Ryan Lei" <ry...@cht.com.tw> wrote:
> >
> > I believe it's a nice idea. A lot of us have experienced the
> > annoying InsufficientCapacityException in Deploying Virtual Machines, but
> > we can't tell exactly why just by reading the logs.
> >
> > If this new API could help us debug this VM deployment process to
> determine
> > exactly which resource is lacking, or probably other internal reasons
> that
> > cause this InsufficientCapacityException, it would be very helpful!
> >
> >
> -------------------------------------------------------------------------------------------
> > Yu-Heng (Ryan) Lei, Associate Researcher
> > Cloud Computing Dept, Chunghwa Telecom Labs
> > ryanlei@cht.com.tw or ryanlei750328@gmail.com
> >
> >
> >
> > On Sat, Feb 1, 2014 at 8:26 AM, Alena Prokharchyk <
> > Alena.Prokharchyk@citrix.com> wrote:
> >
> >> Currently there is no way to know if there is enough resources for vm
> >> deployment, before actual deployVm call is made. The sequence is the
> >> following:
> >>
> >> 1) Deploy Vm is called
> >> 2) DB record is created for the Vm
> >> 3) Storage/Host allocators determine whethere there are enough resources
> >> for vm to be deployed, and return deploy destination to the caller
> stack.
> >> 4) If allocator returns valid deploy destination, VM gets actually
> >> created/started on the backend. If allocators don't return the
> destination,
> >> the DB record created on step 2) gets destroyed, and ResourceAllocation
> >> exception is thrown back to the API caller.
> >>
> >> The API I'm going to introduce, would help you to determine whether CS
> >> physical resources - hosts, storages - can potentially accomodate vm
> >> deployment (considering template/service/diskOffering) at a given time,
> w/o
> >> actually calling the deploy vm. Some admins might find this call useful
> as
> >> they can always make this check before submitting the deployVm, so in
> case
> >> it returns NO, you can fail the deployment immediately, w/o calling
> >> deployVm. Also you can make this call to determine what is lacking for
> >> certain vm deployment, and expand your physical resources accordingly.
> >>
> >> Please let me know if see any pitfalls in the proposal, as well if you
> see
> >> any other use cases that can be solved using this API.
> >>
> >> Prachi, can you please point me to an existing method (or interface)
> >> defined in Allocators code serving this purpose?
> >>
> >> Thanks,
> >> -Alena.
> >>
>
Re: [PROPOSAL] Introduce API returning you an answer from CloudStack storage/host allocators whethere there is enough resources for vm deployment
Posted by abhisek basu <ab...@msn.com>.
Great idea, this would certainly reduce vm deployment failures.
VR including RVR if needs to be created, should get included in this check.
Also, can we include ways to check if a VM can be launched in a certain cluster? The idea is to allow users / admins to be able to launch vm in a certain cluster, but that might be the next functionality.
Sent from my iPhone
> On 1 Feb 2014, at 8:50 pm, "Ryan Lei" <ry...@cht.com.tw> wrote:
>
> I believe it's a nice idea. A lot of us have experienced the
> annoying InsufficientCapacityException in Deploying Virtual Machines, but
> we can't tell exactly why just by reading the logs.
>
> If this new API could help us debug this VM deployment process to determine
> exactly which resource is lacking, or probably other internal reasons that
> cause this InsufficientCapacityException, it would be very helpful!
>
> -------------------------------------------------------------------------------------------
> Yu-Heng (Ryan) Lei, Associate Researcher
> Cloud Computing Dept, Chunghwa Telecom Labs
> ryanlei@cht.com.tw or ryanlei750328@gmail.com
>
>
>
> On Sat, Feb 1, 2014 at 8:26 AM, Alena Prokharchyk <
> Alena.Prokharchyk@citrix.com> wrote:
>
>> Currently there is no way to know if there is enough resources for vm
>> deployment, before actual deployVm call is made. The sequence is the
>> following:
>>
>> 1) Deploy Vm is called
>> 2) DB record is created for the Vm
>> 3) Storage/Host allocators determine whethere there are enough resources
>> for vm to be deployed, and return deploy destination to the caller stack.
>> 4) If allocator returns valid deploy destination, VM gets actually
>> created/started on the backend. If allocators don't return the destination,
>> the DB record created on step 2) gets destroyed, and ResourceAllocation
>> exception is thrown back to the API caller.
>>
>> The API I'm going to introduce, would help you to determine whether CS
>> physical resources - hosts, storages - can potentially accomodate vm
>> deployment (considering template/service/diskOffering) at a given time, w/o
>> actually calling the deploy vm. Some admins might find this call useful as
>> they can always make this check before submitting the deployVm, so in case
>> it returns NO, you can fail the deployment immediately, w/o calling
>> deployVm. Also you can make this call to determine what is lacking for
>> certain vm deployment, and expand your physical resources accordingly.
>>
>> Please let me know if see any pitfalls in the proposal, as well if you see
>> any other use cases that can be solved using this API.
>>
>> Prachi, can you please point me to an existing method (or interface)
>> defined in Allocators code serving this purpose?
>>
>> Thanks,
>> -Alena.
>>
Re: [PROPOSAL] Introduce API returning you an answer from CloudStack
storage/host allocators whethere there is enough resources for vm deployment
Posted by Ryan Lei <ry...@cht.com.tw>.
I believe it's a nice idea. A lot of us have experienced the
annoying InsufficientCapacityException in Deploying Virtual Machines, but
we can't tell exactly why just by reading the logs.
If this new API could help us debug this VM deployment process to determine
exactly which resource is lacking, or probably other internal reasons that
cause this InsufficientCapacityException, it would be very helpful!
-------------------------------------------------------------------------------------------
Yu-Heng (Ryan) Lei, Associate Researcher
Cloud Computing Dept, Chunghwa Telecom Labs
ryanlei@cht.com.tw or ryanlei750328@gmail.com
On Sat, Feb 1, 2014 at 8:26 AM, Alena Prokharchyk <
Alena.Prokharchyk@citrix.com> wrote:
> Currently there is no way to know if there is enough resources for vm
> deployment, before actual deployVm call is made. The sequence is the
> following:
>
> 1) Deploy Vm is called
> 2) DB record is created for the Vm
> 3) Storage/Host allocators determine whethere there are enough resources
> for vm to be deployed, and return deploy destination to the caller stack.
> 4) If allocator returns valid deploy destination, VM gets actually
> created/started on the backend. If allocators don't return the destination,
> the DB record created on step 2) gets destroyed, and ResourceAllocation
> exception is thrown back to the API caller.
>
> The API I'm going to introduce, would help you to determine whether CS
> physical resources - hosts, storages - can potentially accomodate vm
> deployment (considering template/service/diskOffering) at a given time, w/o
> actually calling the deploy vm. Some admins might find this call useful as
> they can always make this check before submitting the deployVm, so in case
> it returns NO, you can fail the deployment immediately, w/o calling
> deployVm. Also you can make this call to determine what is lacking for
> certain vm deployment, and expand your physical resources accordingly.
>
> Please let me know if see any pitfalls in the proposal, as well if you see
> any other use cases that can be solved using this API.
>
> Prachi, can you please point me to an existing method (or interface)
> defined in Allocators code serving this purpose?
>
> Thanks,
> -Alena.
>
Re: [PROPOSAL] Introduce API returning you an answer from CloudStack
storage/host allocators whethere there is enough resources for vm deployment
Posted by Tim Mackey <tm...@gmail.com>.
How deep into dependencies would this go? For example, if a new router VM
was required would that also be checked?
A chunk of the race conditions and state change issues should be resolvable
with ticket reservation system logic, but I'd be concerned the required
locks could create problems during infrastructure failures and with HA
enabled. We ran into a ton of these types of problems implementing the
placement logic in XenServer's workload balancing with HA enabled.
On Sat, Feb 1, 2014 at 12:10 AM, Mike Tutkowski <
mike.tutkowski@solidfire.com> wrote:
> I like the idea in principal. We should fail as fast as is practical.
>
> I wonder, are you just checking if the resources are available? If so, it's
> still possible for deployVM to fail even if this check said all was good
> because the state of the system could change before the actual deployVM is
> started.
>
>
> On Fri, Jan 31, 2014 at 7:20 PM, Madan Ganesh Velayudham <
> madanganesh@me.com
> > wrote:
>
> > Makes lot of sense. Would the same API allow the caller just to check
> (not
> > actually proceed to create) whether resources are present for a
> particular
> > payload? This may be useful for the client who want to proactively check
> > and avoid returning failure.
> >
> > Are there any race condition possibilities when multiple requests are
> > received?
> > ----
> >
> > Madan Ganesh Velayudham P.M.P, C.S.M.,
> > madanganesh@me.com
> >
> >
> >
> >
> > On 01-Feb-2014, at 5:56 am, Alena Prokharchyk <
> > Alena.Prokharchyk@citrix.com> wrote:
> >
> > > Currently there is no way to know if there is enough resources for vm
> > deployment, before actual deployVm call is made. The sequence is the
> > following:
> > >
> > > 1) Deploy Vm is called
> > > 2) DB record is created for the Vm
> > > 3) Storage/Host allocators determine whethere there are enough
> resources
> > for vm to be deployed, and return deploy destination to the caller stack.
> > > 4) If allocator returns valid deploy destination, VM gets actually
> > created/started on the backend. If allocators don't return the
> destination,
> > the DB record created on step 2) gets destroyed, and ResourceAllocation
> > exception is thrown back to the API caller.
> > >
> > > The API I'm going to introduce, would help you to determine whether CS
> > physical resources - hosts, storages - can potentially accomodate vm
> > deployment (considering template/service/diskOffering) at a given time,
> w/o
> > actually calling the deploy vm. Some admins might find this call useful
> as
> > they can always make this check before submitting the deployVm, so in
> case
> > it returns NO, you can fail the deployment immediately, w/o calling
> > deployVm. Also you can make this call to determine what is lacking for
> > certain vm deployment, and expand your physical resources accordingly.
> > >
> > > Please let me know if see any pitfalls in the proposal, as well if you
> > see any other use cases that can be solved using this API.
> > >
> > > Prachi, can you please point me to an existing method (or interface)
> > defined in Allocators code serving this purpose?
> > >
> > > Thanks,
> > > -Alena.
> >
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkowski@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *(tm)*
>
Re: [PROPOSAL] Introduce API returning you an answer from CloudStack
storage/host allocators whethere there is enough resources for vm deployment
Posted by Mike Tutkowski <mi...@solidfire.com>.
I like the idea in principal. We should fail as fast as is practical.
I wonder, are you just checking if the resources are available? If so, it's
still possible for deployVM to fail even if this check said all was good
because the state of the system could change before the actual deployVM is
started.
On Fri, Jan 31, 2014 at 7:20 PM, Madan Ganesh Velayudham <madanganesh@me.com
> wrote:
> Makes lot of sense. Would the same API allow the caller just to check (not
> actually proceed to create) whether resources are present for a particular
> payload? This may be useful for the client who want to proactively check
> and avoid returning failure.
>
> Are there any race condition possibilities when multiple requests are
> received?
> ----
>
> Madan Ganesh Velayudham P.M.P, C.S.M.,
> madanganesh@me.com
>
>
>
>
> On 01-Feb-2014, at 5:56 am, Alena Prokharchyk <
> Alena.Prokharchyk@citrix.com> wrote:
>
> > Currently there is no way to know if there is enough resources for vm
> deployment, before actual deployVm call is made. The sequence is the
> following:
> >
> > 1) Deploy Vm is called
> > 2) DB record is created for the Vm
> > 3) Storage/Host allocators determine whethere there are enough resources
> for vm to be deployed, and return deploy destination to the caller stack.
> > 4) If allocator returns valid deploy destination, VM gets actually
> created/started on the backend. If allocators don't return the destination,
> the DB record created on step 2) gets destroyed, and ResourceAllocation
> exception is thrown back to the API caller.
> >
> > The API I'm going to introduce, would help you to determine whether CS
> physical resources - hosts, storages - can potentially accomodate vm
> deployment (considering template/service/diskOffering) at a given time, w/o
> actually calling the deploy vm. Some admins might find this call useful as
> they can always make this check before submitting the deployVm, so in case
> it returns NO, you can fail the deployment immediately, w/o calling
> deployVm. Also you can make this call to determine what is lacking for
> certain vm deployment, and expand your physical resources accordingly.
> >
> > Please let me know if see any pitfalls in the proposal, as well if you
> see any other use cases that can be solved using this API.
> >
> > Prachi, can you please point me to an existing method (or interface)
> defined in Allocators code serving this purpose?
> >
> > Thanks,
> > -Alena.
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*(tm)*
Re: [PROPOSAL] Introduce API returning you an answer from CloudStack
storage/host allocators whethere there is enough resources for vm deployment
Posted by Madan Ganesh Velayudham <ma...@me.com>.
Makes lot of sense. Would the same API allow the caller just to check (not actually proceed to create) whether resources are present for a particular payload? This may be useful for the client who want to proactively check and avoid returning failure.
Are there any race condition possibilities when multiple requests are received?
----
Madan Ganesh Velayudham P.M.P, C.S.M.,
madanganesh@me.com
On 01-Feb-2014, at 5:56 am, Alena Prokharchyk <Al...@citrix.com> wrote:
> Currently there is no way to know if there is enough resources for vm deployment, before actual deployVm call is made. The sequence is the following:
>
> 1) Deploy Vm is called
> 2) DB record is created for the Vm
> 3) Storage/Host allocators determine whethere there are enough resources for vm to be deployed, and return deploy destination to the caller stack.
> 4) If allocator returns valid deploy destination, VM gets actually created/started on the backend. If allocators don't return the destination, the DB record created on step 2) gets destroyed, and ResourceAllocation exception is thrown back to the API caller.
>
> The API I'm going to introduce, would help you to determine whether CS physical resources - hosts, storages - can potentially accomodate vm deployment (considering template/service/diskOffering) at a given time, w/o actually calling the deploy vm. Some admins might find this call useful as they can always make this check before submitting the deployVm, so in case it returns NO, you can fail the deployment immediately, w/o calling deployVm. Also you can make this call to determine what is lacking for certain vm deployment, and expand your physical resources accordingly.
>
> Please let me know if see any pitfalls in the proposal, as well if you see any other use cases that can be solved using this API.
>
> Prachi, can you please point me to an existing method (or interface) defined in Allocators code serving this purpose?
>
> Thanks,
> -Alena.
Re: [PROPOSAL] Introduce API returning you an answer from
CloudStack storage/host allocators whethere there is enough resources for vm
deployment
Posted by Alena Prokharchyk <Al...@citrix.com>.
On 2/3/14, 1:07 PM, "David Nalley" <da...@gnsa.us> wrote:
>> After reviewing all the suggestions, we should probably do this in the
>> future:
>>
>> 1) Introduce the call reserveCapacityForVirtualMachine. You can execute
>> the call with "count" parameter to reserve the capacity for "n" vms w/o
>> actually deploying them on the backend(no actual volume creation on
>> storage, no vm start on the host).
>
>Doesn't the deployVirtualMachine with startvm=false accomplish most of
>this? (admittedly there'd be volume creation, and vm start)
No, it doesn¹t. deployVm with startVm=false just allocates the volume/vm
in the DB. Allocators are being invoked only when vm starts for the first
time.
>
>> 2) Enhance deployVm call with "count" parameter. Before actual vm
>> deployment, reserveCapacity underlying method should be called to
>>reserve
>> the capacity for all "n" vms requested in the call.
>
>I think you'll find this is a heavily desired functionality. I know
>folks who have rather simple portals that call deployVM in a for loop
>to pull this off.
>
>> 3) Improve InsufficientCapacity error messages returned when deployVm
>> fails to return more clear reason for vm deployment failure.
>
>I'd be somewhat worried about information leakage even more with
>ability to use count=n. Imagine me going to $foo_cloud_provider and
>running this to determine exactly how much capacity is available. I
>assume (though didn't see it explicitly called out) that quotas would
>cause a failure in capacity checking as well? If so, that would
>probably ameliorate any concerns.
Our API layer generates error messages based on the caller context. Of
course, the detailed message will be returned only to the Root admin while
regular user will get the generic message (the way we do now)
>
>--David
Re: [PROPOSAL] Introduce API returning you an answer from CloudStack
storage/host allocators whethere there is enough resources for vm deployment
Posted by David Nalley <da...@gnsa.us>.
> After reviewing all the suggestions, we should probably do this in the
> future:
>
> 1) Introduce the call reserveCapacityForVirtualMachine. You can execute
> the call with "count" parameter to reserve the capacity for "n" vms w/o
> actually deploying them on the backend(no actual volume creation on
> storage, no vm start on the host).
Doesn't the deployVirtualMachine with startvm=false accomplish most of
this? (admittedly there'd be volume creation, and vm start)
> 2) Enhance deployVm call with "count" parameter. Before actual vm
> deployment, reserveCapacity underlying method should be called to reserve
> the capacity for all "n" vms requested in the call.
I think you'll find this is a heavily desired functionality. I know
folks who have rather simple portals that call deployVM in a for loop
to pull this off.
> 3) Improve InsufficientCapacity error messages returned when deployVm
> fails to return more clear reason for vm deployment failure.
I'd be somewhat worried about information leakage even more with
ability to use count=n. Imagine me going to $foo_cloud_provider and
running this to determine exactly how much capacity is available. I
assume (though didn't see it explicitly called out) that quotas would
cause a failure in capacity checking as well? If so, that would
probably ameliorate any concerns.
--David
Re: [PROPOSAL] Introduce API returning you an answer from
CloudStack storage/host allocators whethere there is enough resources for vm
deployment
Posted by Alena Prokharchyk <Al...@citrix.com>.
Thank you everyone for responding with your concerns and ideas. After
analyzing all the input and talking to Alex, I would like to recall the
original proposal as:
1) The call doesn't actually do any resource reservation, there is no
guarantee that subsequent VM call wouldn't fail if executed some time
later.
2) It's not very useful to the check for a SINGLE vm, only to deploy this
single vm later. What would make more sense - to check if we can
accomodate "n" number of vms, and may be even to reserve the capacity for
them.
After reviewing all the suggestions, we should probably do this in the
future:
1) Introduce the call reserveCapacityForVirtualMachine. You can execute
the call with "count" parameter to reserve the capacity for "n" vms w/o
actually deploying them on the backend(no actual volume creation on
storage, no vm start on the host).
2) Enhance deployVm call with "count" parameter. Before actual vm
deployment, reserveCapacity underlying method should be called to reserve
the capacity for all "n" vms requested in the call.
3) Improve InsufficientCapacity error messages returned when deployVm
fails to return more clear reason for vm deployment failure.
Let me know what you all think,
-Alena.
On 2/3/14, 9:55 AM, "Alex Huang" <Al...@citrix.com> wrote:
>What's really needed is a reserve API. It would be a big undertaking
>though as that concept is not in any of the network and storage plugins.
>
>--alex
>
>> -----Original Message-----
>> From: David Nalley [mailto:david@gnsa.us]
>> Sent: Monday, February 3, 2014 6:10 AM
>> To: dev@cloudstack.apache.org
>> Cc: Prachi Damle
>> Subject: Re: [PROPOSAL] Introduce API returning you an answer from
>> CloudStack storage/host allocators whethere there is enough resources
>>for
>> vm deployment
>>
>> Hi Alena:
>>
>> I am not sure that I see a ton of value in this - and let me explain
>>why:
>>
>> 1. This could potentially be an information disclosure issue - Could I
>> potentially see how much storage or other resources are available.
>> Does
>>
>> 2. At any scale things are going to potentially change rapidly, so a
>>delay of 5
>> minutes may mean that the content is outdated and no longer valid.
>>
>> 3. Most importantly deploying a single machine isn't interesting, and
>>why
>> would I bother checking. (And this also brings up one of the things I
>>find
>> lacking in the deployVM API call.) What I'd be interested in knowing is
>> whether I could deploy 5, 50, or 500 machines. Perhaps adding a count
>> parameter to the deployVirtualMachine would help (and the corresponding
>> API call to check that resources are
>> available.)
>>
>> --David
>>
>> On Fri, Jan 31, 2014 at 7:26 PM, Alena Prokharchyk
>> <Al...@citrix.com> wrote:
>> > Currently there is no way to know if there is enough resources for vm
>> deployment, before actual deployVm call is made. The sequence is the
>> following:
>> >
>> > 1) Deploy Vm is called
>> > 2) DB record is created for the Vm
>> > 3) Storage/Host allocators determine whethere there are enough
>> resources for vm to be deployed, and return deploy destination to the
>>caller
>> stack.
>> > 4) If allocator returns valid deploy destination, VM gets actually
>> created/started on the backend. If allocators don't return the
>>destination,
>> the DB record created on step 2) gets destroyed, and ResourceAllocation
>> exception is thrown back to the API caller.
>> >
>> > The API I'm going to introduce, would help you to determine whether CS
>> physical resources - hosts, storages - can potentially accomodate vm
>> deployment (considering template/service/diskOffering) at a given time,
>> w/o actually calling the deploy vm. Some admins might find this call
>>useful as
>> they can always make this check before submitting the deployVm, so in
>>case
>> it returns NO, you can fail the deployment immediately, w/o calling
>> deployVm. Also you can make this call to determine what is lacking for
>>certain
>> vm deployment, and expand your physical resources accordingly.
>> >
>> > Please let me know if see any pitfalls in the proposal, as well if
>>you see any
>> other use cases that can be solved using this API.
>> >
>> > Prachi, can you please point me to an existing method (or interface)
>> defined in Allocators code serving this purpose?
>> >
>> > Thanks,
>> > -Alena.
RE: [PROPOSAL] Introduce API returning you an answer from
CloudStack storage/host allocators whethere there is enough resources for vm
deployment
Posted by Alex Huang <Al...@citrix.com>.
What's really needed is a reserve API. It would be a big undertaking though as that concept is not in any of the network and storage plugins.
--alex
> -----Original Message-----
> From: David Nalley [mailto:david@gnsa.us]
> Sent: Monday, February 3, 2014 6:10 AM
> To: dev@cloudstack.apache.org
> Cc: Prachi Damle
> Subject: Re: [PROPOSAL] Introduce API returning you an answer from
> CloudStack storage/host allocators whethere there is enough resources for
> vm deployment
>
> Hi Alena:
>
> I am not sure that I see a ton of value in this - and let me explain why:
>
> 1. This could potentially be an information disclosure issue - Could I
> potentially see how much storage or other resources are available.
> Does
>
> 2. At any scale things are going to potentially change rapidly, so a delay of 5
> minutes may mean that the content is outdated and no longer valid.
>
> 3. Most importantly deploying a single machine isn't interesting, and why
> would I bother checking. (And this also brings up one of the things I find
> lacking in the deployVM API call.) What I'd be interested in knowing is
> whether I could deploy 5, 50, or 500 machines. Perhaps adding a count
> parameter to the deployVirtualMachine would help (and the corresponding
> API call to check that resources are
> available.)
>
> --David
>
> On Fri, Jan 31, 2014 at 7:26 PM, Alena Prokharchyk
> <Al...@citrix.com> wrote:
> > Currently there is no way to know if there is enough resources for vm
> deployment, before actual deployVm call is made. The sequence is the
> following:
> >
> > 1) Deploy Vm is called
> > 2) DB record is created for the Vm
> > 3) Storage/Host allocators determine whethere there are enough
> resources for vm to be deployed, and return deploy destination to the caller
> stack.
> > 4) If allocator returns valid deploy destination, VM gets actually
> created/started on the backend. If allocators don't return the destination,
> the DB record created on step 2) gets destroyed, and ResourceAllocation
> exception is thrown back to the API caller.
> >
> > The API I'm going to introduce, would help you to determine whether CS
> physical resources - hosts, storages - can potentially accomodate vm
> deployment (considering template/service/diskOffering) at a given time,
> w/o actually calling the deploy vm. Some admins might find this call useful as
> they can always make this check before submitting the deployVm, so in case
> it returns NO, you can fail the deployment immediately, w/o calling
> deployVm. Also you can make this call to determine what is lacking for certain
> vm deployment, and expand your physical resources accordingly.
> >
> > Please let me know if see any pitfalls in the proposal, as well if you see any
> other use cases that can be solved using this API.
> >
> > Prachi, can you please point me to an existing method (or interface)
> defined in Allocators code serving this purpose?
> >
> > Thanks,
> > -Alena.
Re: [PROPOSAL] Introduce API returning you an answer from CloudStack
storage/host allocators whethere there is enough resources for vm deployment
Posted by Prasanna Santhanam <ts...@apache.org>.
On Mon, Feb 03, 2014 at 09:09:44AM -0500, David Nalley wrote:
> 3. Most importantly deploying a single machine isn't interesting, and
> why would I bother checking. (And this also brings up one of the
> things I find lacking in the deployVM API call.) What I'd be
> interested in knowing is whether I could deploy 5, 50, or 500
> machines. Perhaps adding a count parameter to the deployVirtualMachine
> would help (and the corresponding API call to check that resources are
> available.)
+1 - incredibly useful for large hadoop clusters.
--
Prasanna.,
------------------------
Powered by BigRock.com
Re: [PROPOSAL] Introduce API returning you an answer from CloudStack
storage/host allocators whethere there is enough resources for vm deployment
Posted by David Nalley <da...@gnsa.us>.
Hi Alena:
I am not sure that I see a ton of value in this - and let me explain why:
1. This could potentially be an information disclosure issue - Could I
potentially see how much storage or other resources are available.
Does
2. At any scale things are going to potentially change rapidly, so a
delay of 5 minutes may mean that the content is outdated and no longer
valid.
3. Most importantly deploying a single machine isn't interesting, and
why would I bother checking. (And this also brings up one of the
things I find lacking in the deployVM API call.) What I'd be
interested in knowing is whether I could deploy 5, 50, or 500
machines. Perhaps adding a count parameter to the deployVirtualMachine
would help (and the corresponding API call to check that resources are
available.)
--David
On Fri, Jan 31, 2014 at 7:26 PM, Alena Prokharchyk
<Al...@citrix.com> wrote:
> Currently there is no way to know if there is enough resources for vm deployment, before actual deployVm call is made. The sequence is the following:
>
> 1) Deploy Vm is called
> 2) DB record is created for the Vm
> 3) Storage/Host allocators determine whethere there are enough resources for vm to be deployed, and return deploy destination to the caller stack.
> 4) If allocator returns valid deploy destination, VM gets actually created/started on the backend. If allocators don't return the destination, the DB record created on step 2) gets destroyed, and ResourceAllocation exception is thrown back to the API caller.
>
> The API I'm going to introduce, would help you to determine whether CS physical resources - hosts, storages - can potentially accomodate vm deployment (considering template/service/diskOffering) at a given time, w/o actually calling the deploy vm. Some admins might find this call useful as they can always make this check before submitting the deployVm, so in case it returns NO, you can fail the deployment immediately, w/o calling deployVm. Also you can make this call to determine what is lacking for certain vm deployment, and expand your physical resources accordingly.
>
> Please let me know if see any pitfalls in the proposal, as well if you see any other use cases that can be solved using this API.
>
> Prachi, can you please point me to an existing method (or interface) defined in Allocators code serving this purpose?
>
> Thanks,
> -Alena.