You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Nitin Mehta <Ni...@citrix.com> on 2013/03/08 13:31:45 UTC
Re: [DISCUSS] Scaling up CPU and RAM for running VMs
Alex - Thanks very much for pointing out earlier this week that for
scaling up the vm we shouldn't change the vm lifecycle. I also read
http://markmail.org/message/6c6njactsklot62h
and understand that scaling up a vm is a vm operation and shouldn't be
mixed with vm lifecycle. So as you suggested in the thread that if I need
to prevent other
Vm operations happening during this operation I would need to introduce an
attribute
1. For this I would need to introduce a column in vm_instance table which
would be set during scale up operation.
2. To prevent other operations from happening this attribute needs to be
checked in all the other vm operations. There is no single common piece of
code where I can put the check so I have to explicitly check for this
attribute in all the operations code right ? I see that for "vm snapshot"
operation we have put this check in vm state transition method but this
method is called only for vm lifecycle changes. So when "vm snapshot"
happens the user might also scale up the vm. There might be a need for
them to be exclusive.
3. If I need to say lock capacity before the operation and modify it after
the operation is done (say during failure) how do I do it w/o coupling the
code changes or is it ok for now to do so ?
Thanks,
-Nitin
On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
>Hi Nitin,
>
>Please see below
>
>Hari
>
>-----Original Message-----
>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
>Sent: Tuesday, February 12, 2013 7:15 AM
>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
>Abhinandan Prateek
>Cc: Chip Childers
>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>
>Apologize for the delayed response. Was involved in other issues.
>Please find answers inline.
>
>>
>>-----Original Message-----
>>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
>>Sent: Thursday, January 24, 2013 12:26 PM
>>To: cloudstack-dev@incubator.apache.org
>>Cc: Nitin Mehta
>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>>
>>Hi Nitin,
>>I am planning to take the QA job for this feature. Have reviewed the
>>functional spec, gone through community discussion and have the
>>following questions
>>
>>1-What is expected behavior of CS for Operating systems which do not
>>support dynamic scaling . ?
>
>Will throw a not supported exception
>
>Hari: How do we know which OS is supported or not? Is it going to be part
>of the "capabilities" of hypervisor? Or where will this be
>specified/configured?
>PS: I know we plan to implement this only on VMware for now, but when
>installed/shipped, how will CS know the supported Hypervisor/OS?
>
>>
>>2-How much resources can be scaled up, is it limited by availability of
>>resource on host .?
>>
>>[Koushik Das ]
>>"Having a range for CPU/RAM in compute offering is definitely another
>>way of doing it. But creating the higher limit would be tricky. I am
>>not sure if it is always known to users how much they want to scale up
>>to at the time of deploying VM. Moreover if the higher limit is known
>>then the VM can be deployed with that value itself. Also in case of
>>having a range in the offering the usage part needs to be handled
>>appropriately. Currently usage is purely based on the offering and
>>individual values are not stored".
>>[/Koushik Das]
>
>It is not limited by the resources on host but on the available capacity
>in any of the host within the cluster.
>
>Hari: I'm not sure I understand the question - how is this any different
>than requesting a new VM? Or upgrading from offering A to Offering B that
>exists today (although VM needs to be shutdown)?
>
>>
>>it seems its totally depend on service offering , please correct me if
>>I am wrong.
>>
>>3- Scheduled snapshot of volumes during the operation .
>>
>>[NITIN]
>>For vmware, the entire vm is locked by HV and this can be an issue. I
>>will leverage on current implementations for existing interactions like
>>scheduled snapshots events during live migration and will replicate the
>>same.
>>[/NITIN]
>>
>>Can you elaborate what is expected in case of VMware .
>
>What I mean is there is an existing functionality which is implemented
>the same way. I will just do it the same way.
>
>>
>>4 - what is expected behavior in case of powers off the vm during the
>>operation .? is it different for different hypervisors.?
>
>There is nothing much to do for powered off vms because we will just
>update the DB. When the vm is started it will pick up these values from
>the DB.
>This functionality already exists.
>
>>
>>5- what is expected in case of migration fails( In FS no description
>>about this),
>> -CS will retry to migrate it again if yes how many time ?
>> - will it mark as a failure and can't scale up(even resources
>>are available in cluster) ?
>
>We will retry N (configurable) times and if unsuccessful we will throw an
>exception to the user.
>
>Hari: Can you please elaborate why a migration might fail? And, is the
>"N" configurable times a new global?
>
>>
>>6- Apart from "scaleVirtualMachine" any other APIs are getting
>>changed ?
>
>No
>
>>
>>7-Scale down is allowed ? (still open issue in FS)
>
>No for time being.
>
>>
>>8-Are we going to introduce custom compute offering (still open issue
>>in
>>FS) ?
>
>No for now
>
>>
>>9- what are the guide line for upgrade ?
>
>There is nothing for upgrade because we do not introduce new values in DB.
>
>>
>>10-Any DB changes ?
>
>See #9 above.
>
>>
>>11- which Usage events are getting introduced for billing .?
>
>Will update the FS.
>
>>
>>12-hypervisor support ,is it only for VMware (as per FS) or its
>>getting extended for XS/KVM also ?
>
>There are subtasks opened for XS and KVM. I am doing it only for Vmware.
>
>>
>>Thanks
>>Prashant Kumar Mishra
>>
>>
>>-----Original Message-----
>>From: Koushik Das [mailto:koushik.das@citrix.com]
>>Sent: Saturday, December 15, 2012 11:14 PM
>>To: cloudstack-dev@incubator.apache.org
>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>
>>Currently CS supports changing CPU and RAM for stopped VM. This is
>>achieved by changing compute offering of the VM (with new CPU and RAM
>>values) and then starting it. I am planning to extend the same for
>>running VM as well. Initially planning to do it for Vmware where CPU
>>and RAM can be dynamically increased. Support of other HVs can also be
>>added if they support increasing CPU/RAM.
>>
>>Assuming that in the updated compute offering only CPU and RAM has
>>changed, the deployment planner can either select the same host in
>>which case the values are dynamically scaled up OR a different one in
>>which case the operation fails. In future if there is support for live
>>migration (provided HV supports it) then another option in the latter
>>case could be to migrate the VM first and then scale it up.
>>
>>I will start working on the FS and share it out sometime next week.
>>
>>Comments/suggestions?
>>
>>Thanks,
>>Koushik
>
Re: [DISCUSS] Scaling up CPU and RAM for running VMs
Posted by Mice Xia <we...@gmail.com>.
Nitin & Kelven,
I haven't updated the codes with syncAsyncJobExecution, and the conclusion
is making all vm api synchronized? I'm ok with this change, currently, for
a single vm instance, few apis can work on it concurrently.
-Mice
On Fri, Mar 15, 2013 at 12:20 PM, Nitin Mehta <Ni...@citrix.com>wrote:
> Thanks Kelven.
> I saw that, but making this change would mean that all the vm API's -
> deployvm, start-stop, reboot, destroy vm etc.. will now be synchronized
> with this change.
>
> Hope you guys are ok with this ??
>
> Thanks,
> -Nitin
>
> On 13/03/13 11:31 PM, "Kelven Yang" <ke...@citrix.com> wrote:
>
> >Nitin,
> >
> >Sorry to reply late, have been busy working on a patch release for
> >customer.
> >
> >AsyncJob manager does provide a mechanism that you can do synchronized the
> >job execution against a object. You may check out
> >AsyncJobManager.syncAsyncJobExecution().
> >
> >Kelven
> >
> >On 3/12/13 11:27 AM, "Nitin Mehta" <Ni...@citrix.com> wrote:
> >>syncAsyncJobExecutionThanks Alex.
> >>
> >>Kelven / Alex - I took a look into this and it seems that the framework
> >>has the capability to do this and this would solve the problem for
> >>scaling
> >>up vm and vm snapshots.
> >>But, we currently don't do synchronization for the vm object. Doing this
> >>means all the vm operations will be syncronized now. Are we fine with
> >>that
> >>?
> >>
> >>Thanks,
> >>-Nitin
> >>
> >>On 09/03/13 10:36 PM, "Alex Huang" <Al...@citrix.com> wrote:
> >>
> >>>Nitin,
> >>>
> >>>The other approach to this is to utilize the syncing feature in the job
> >>>queue. I've cced Kelven to see if he can give you more detail. His
> >>>code
> >>>is capable of syncing operations on a single object so you don't have to
> >>>add processing states.
> >>>
> >>>Given that all of your operations and vm snapshot operations must have
> >>>come in through the job queue, you might already have that ability to
> >>>not
> >>>interfere with each other.
> >>>
> >>>VM States are different because there can be outside changes (through
> >>>other vm managers) that cause vm life cycle to behave differently.
> >>>
> >>>--Alex
> >>>
> >>>> -----Original Message-----
> >>>> From: Nitin Mehta
> >>>> Sent: Saturday, March 9, 2013 2:35 AM
> >>>> To: cloudstack-dev@incubator.apache.org; Prashant Kumar Mishra;
> >>>> Abhinandan Prateek; Alex Huang
> >>>> Cc: Chip Childers
> >>>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>>
> >>>> Hi Alex,
> >>>> I had one more question. Say the MS is shut down or restarted, when do
> >>>>we
> >>>> clear this attribute in this case ?
> >>>>
> >>>> On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
> >>>>
> >>>> >Alex - Thanks very much for pointing out earlier this week that for
> >>>> >scaling up the vm we shouldn't change the vm lifecycle. I also read
> >>>> >http://markmail.org/message/6c6njactsklot62h
> >>>> >and understand that scaling up a vm is a vm operation and shouldn't
> >>>>be
> >>>> >mixed with vm lifecycle. So as you suggested in the thread that if I
> >>>> >need to prevent other Vm operations happening during this operation I
> >>>> >would need to introduce an attribute
> >>>> >
> >>>> >1. For this I would need to introduce a column in vm_instance table
> >>>> >which would be set during scale up operation.
> >>>> >2. To prevent other operations from happening this attribute needs to
> >>>> >be checked in all the other vm operations. There is no single common
> >>>> >piece of code where I can put the check so I have to explicitly check
> >>>> >for this attribute in all the operations code right ? I see that for
> >>>>"vm
> >>>> snapshot"
> >>>> >operation we have put this check in vm state transition method but
> >>>>this
> >>>> >method is called only for vm lifecycle changes. So when "vm snapshot"
> >>>> >happens the user might also scale up the vm. There might be a need
> >>>>for
> >>>> >them to be exclusive.
> >>>> >3. If I need to say lock capacity before the operation and modify it
> >>>> >after the operation is done (say during failure) how do I do it w/o
> >>>> >coupling the code changes or is it ok for now to do so ?
> >>>> >
> >>>> >
> >>>> >Thanks,
> >>>> >-Nitin
> >>>> >
> >>>> >On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
> >>>> >
> >>>> >>Hi Nitin,
> >>>> >>
> >>>> >>Please see below
> >>>> >>
> >>>> >>Hari
> >>>> >>
> >>>> >>-----Original Message-----
> >>>> >>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
> >>>> >>Sent: Tuesday, February 12, 2013 7:15 AM
> >>>> >>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
> >>>> >>Abhinandan Prateek
> >>>> >>Cc: Chip Childers
> >>>> >>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>> >>
> >>>> >>Apologize for the delayed response. Was involved in other issues.
> >>>> >>Please find answers inline.
> >>>> >>
> >>>> >>>
> >>>> >>>-----Original Message-----
> >>>> >>>From: Prashant Kumar Mishra
> >>>>[mailto:prashantkumar.mishra@citrix.com]
> >>>> >>>Sent: Thursday, January 24, 2013 12:26 PM
> >>>> >>>To: cloudstack-dev@incubator.apache.org
> >>>> >>>Cc: Nitin Mehta
> >>>> >>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>> >>>
> >>>> >>>Hi Nitin,
> >>>> >>>I am planning to take the QA job for this feature. Have reviewed
> >>>>the
> >>>> >>>functional spec, gone through community discussion and have the
> >>>> >>>following questions
> >>>> >>>
> >>>> >>>1-What is expected behavior of CS for Operating systems which do
> >>>>not
> >>>> >>>support dynamic scaling . ?
> >>>> >>
> >>>> >>Will throw a not supported exception
> >>>> >>
> >>>> >>Hari: How do we know which OS is supported or not? Is it going to be
> >>>> >>part of the "capabilities" of hypervisor? Or where will this be
> >>>> >>specified/configured?
> >>>> >>PS: I know we plan to implement this only on VMware for now, but
> >>>>when
> >>>> >>installed/shipped, how will CS know the supported Hypervisor/OS?
> >>>> >>
> >>>> >>>
> >>>> >>>2-How much resources can be scaled up, is it limited by
> >>>>availability
> >>>> >>>of resource on host .?
> >>>> >>>
> >>>> >>>[Koushik Das ]
> >>>> >>>"Having a range for CPU/RAM in compute offering is definitely
> >>>>another
> >>>> >>>way of doing it. But creating the higher limit would be tricky. I
> >>>>am
> >>>> >>>not sure if it is always known to users how much they want to scale
> >>>> >>>up to at the time of deploying VM. Moreover if the higher limit is
> >>>> >>>known then the VM can be deployed with that value itself. Also in
> >>>> >>>case of having a range in the offering the usage part needs to be
> >>>> >>>handled appropriately. Currently usage is purely based on the
> >>>> >>>offering and individual values are not stored".
> >>>> >>>[/Koushik Das]
> >>>> >>
> >>>> >>It is not limited by the resources on host but on the available
> >>>> >>capacity in any of the host within the cluster.
> >>>> >>
> >>>> >>Hari: I'm not sure I understand the question - how is this any
> >>>> >>different than requesting a new VM? Or upgrading from offering A to
> >>>> >>Offering B that exists today (although VM needs to be shutdown)?
> >>>> >>
> >>>> >>>
> >>>> >>>it seems its totally depend on service offering , please correct
> >>>>me
> >>>> >>>if I am wrong.
> >>>> >>>
> >>>> >>>3- Scheduled snapshot of volumes during the operation .
> >>>> >>>
> >>>> >>>[NITIN]
> >>>> >>>For vmware, the entire vm is locked by HV and this can be an issue.
> >>>>I
> >>>> >>>will leverage on current implementations for existing interactions
> >>>> >>>like scheduled snapshots events during live migration and will
> >>>> >>>replicate the same.
> >>>> >>>[/NITIN]
> >>>> >>>
> >>>> >>>Can you elaborate what is expected in case of VMware .
> >>>> >>
> >>>> >>What I mean is there is an existing functionality which is
> >>>>implemented
> >>>> >>the same way. I will just do it the same way.
> >>>> >>
> >>>> >>>
> >>>> >>>4 - what is expected behavior in case of powers off the vm during
> >>>> >>>the operation .? is it different for different hypervisors.?
> >>>> >>
> >>>> >>There is nothing much to do for powered off vms because we will just
> >>>> >>update the DB. When the vm is started it will pick up these values
> >>>> >>from the DB.
> >>>> >>This functionality already exists.
> >>>> >>
> >>>> >>>
> >>>> >>>5- what is expected in case of migration fails( In FS no
> >>>>description
> >>>> >>>about this),
> >>>> >>> -CS will retry to migrate it again if yes how many time ?
> >>>> >>> - will it mark as a failure and can't scale up(even
> >>>>resources
> >>>> >>>are available in cluster) ?
> >>>> >>
> >>>> >>We will retry N (configurable) times and if unsuccessful we will
> >>>>throw
> >>>> >>an exception to the user.
> >>>> >>
> >>>> >>Hari: Can you please elaborate why a migration might fail? And, is
> >>>>the
> >>>> >>"N" configurable times a new global?
> >>>> >>
> >>>> >>>
> >>>> >>>6- Apart from "scaleVirtualMachine" any other APIs are getting
> >>>> >>>changed ?
> >>>> >>
> >>>> >>No
> >>>> >>
> >>>> >>>
> >>>> >>>7-Scale down is allowed ? (still open issue in FS)
> >>>> >>
> >>>> >>No for time being.
> >>>> >>
> >>>> >>>
> >>>> >>>8-Are we going to introduce custom compute offering (still open
> >>>>issue
> >>>> >>>in
> >>>> >>>FS) ?
> >>>> >>
> >>>> >>No for now
> >>>> >>
> >>>> >>>
> >>>> >>>9- what are the guide line for upgrade ?
> >>>> >>
> >>>> >>There is nothing for upgrade because we do not introduce new values
> >>>>in
> >>>> >>DB.
> >>>> >>
> >>>> >>>
> >>>> >>>10-Any DB changes ?
> >>>> >>
> >>>> >>See #9 above.
> >>>> >>
> >>>> >>>
> >>>> >>>11- which Usage events are getting introduced for billing .?
> >>>> >>
> >>>> >>Will update the FS.
> >>>> >>
> >>>> >>>
> >>>> >>>12-hypervisor support ,is it only for VMware (as per FS) or its
> >>>> >>>getting extended for XS/KVM also ?
> >>>> >>
> >>>> >>There are subtasks opened for XS and KVM. I am doing it only for
> >>>>Vmware.
> >>>> >>
> >>>> >>>
> >>>> >>>Thanks
> >>>> >>>Prashant Kumar Mishra
> >>>> >>>
> >>>> >>>
> >>>> >>>-----Original Message-----
> >>>> >>>From: Koushik Das [mailto:koushik.das@citrix.com]
> >>>> >>>Sent: Saturday, December 15, 2012 11:14 PM
> >>>> >>>To: cloudstack-dev@incubator.apache.org
> >>>> >>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>> >>>
> >>>> >>>Currently CS supports changing CPU and RAM for stopped VM. This is
> >>>> >>>achieved by changing compute offering of the VM (with new CPU and
> >>>> RAM
> >>>> >>>values) and then starting it. I am planning to extend the same for
> >>>> >>>running VM as well. Initially planning to do it for Vmware where
> >>>>CPU
> >>>> >>>and RAM can be dynamically increased. Support of other HVs can also
> >>>> >>>be added if they support increasing CPU/RAM.
> >>>> >>>
> >>>> >>>Assuming that in the updated compute offering only CPU and RAM has
> >>>> >>>changed, the deployment planner can either select the same host in
> >>>> >>>which case the values are dynamically scaled up OR a different one
> >>>>in
> >>>> >>>which case the operation fails. In future if there is support for
> >>>> >>>live migration (provided HV supports it) then another option in the
> >>>> >>>latter case could be to migrate the VM first and then scale it up.
> >>>> >>>
> >>>> >>>I will start working on the FS and share it out sometime next week.
> >>>> >>>
> >>>> >>>Comments/suggestions?
> >>>> >>>
> >>>> >>>Thanks,
> >>>> >>>Koushik
> >>>> >>
> >>>> >
> >>>
> >>
> >
>
>
Re: [DISCUSS] Scaling up CPU and RAM for running VMs
Posted by Nitin Mehta <Ni...@citrix.com>.
Thanks Kelven.
I saw that, but making this change would mean that all the vm API's -
deployvm, start-stop, reboot, destroy vm etc.. will now be synchronized
with this change.
Hope you guys are ok with this ??
Thanks,
-Nitin
On 13/03/13 11:31 PM, "Kelven Yang" <ke...@citrix.com> wrote:
>Nitin,
>
>Sorry to reply late, have been busy working on a patch release for
>customer.
>
>AsyncJob manager does provide a mechanism that you can do synchronized the
>job execution against a object. You may check out
>AsyncJobManager.syncAsyncJobExecution().
>
>Kelven
>
>On 3/12/13 11:27 AM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>>syncAsyncJobExecutionThanks Alex.
>>
>>Kelven / Alex - I took a look into this and it seems that the framework
>>has the capability to do this and this would solve the problem for
>>scaling
>>up vm and vm snapshots.
>>But, we currently don't do synchronization for the vm object. Doing this
>>means all the vm operations will be syncronized now. Are we fine with
>>that
>>?
>>
>>Thanks,
>>-Nitin
>>
>>On 09/03/13 10:36 PM, "Alex Huang" <Al...@citrix.com> wrote:
>>
>>>Nitin,
>>>
>>>The other approach to this is to utilize the syncing feature in the job
>>>queue. I've cced Kelven to see if he can give you more detail. His
>>>code
>>>is capable of syncing operations on a single object so you don't have to
>>>add processing states.
>>>
>>>Given that all of your operations and vm snapshot operations must have
>>>come in through the job queue, you might already have that ability to
>>>not
>>>interfere with each other.
>>>
>>>VM States are different because there can be outside changes (through
>>>other vm managers) that cause vm life cycle to behave differently.
>>>
>>>--Alex
>>>
>>>> -----Original Message-----
>>>> From: Nitin Mehta
>>>> Sent: Saturday, March 9, 2013 2:35 AM
>>>> To: cloudstack-dev@incubator.apache.org; Prashant Kumar Mishra;
>>>> Abhinandan Prateek; Alex Huang
>>>> Cc: Chip Childers
>>>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>>
>>>> Hi Alex,
>>>> I had one more question. Say the MS is shut down or restarted, when do
>>>>we
>>>> clear this attribute in this case ?
>>>>
>>>> On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>>>>
>>>> >Alex - Thanks very much for pointing out earlier this week that for
>>>> >scaling up the vm we shouldn't change the vm lifecycle. I also read
>>>> >http://markmail.org/message/6c6njactsklot62h
>>>> >and understand that scaling up a vm is a vm operation and shouldn't
>>>>be
>>>> >mixed with vm lifecycle. So as you suggested in the thread that if I
>>>> >need to prevent other Vm operations happening during this operation I
>>>> >would need to introduce an attribute
>>>> >
>>>> >1. For this I would need to introduce a column in vm_instance table
>>>> >which would be set during scale up operation.
>>>> >2. To prevent other operations from happening this attribute needs to
>>>> >be checked in all the other vm operations. There is no single common
>>>> >piece of code where I can put the check so I have to explicitly check
>>>> >for this attribute in all the operations code right ? I see that for
>>>>"vm
>>>> snapshot"
>>>> >operation we have put this check in vm state transition method but
>>>>this
>>>> >method is called only for vm lifecycle changes. So when "vm snapshot"
>>>> >happens the user might also scale up the vm. There might be a need
>>>>for
>>>> >them to be exclusive.
>>>> >3. If I need to say lock capacity before the operation and modify it
>>>> >after the operation is done (say during failure) how do I do it w/o
>>>> >coupling the code changes or is it ok for now to do so ?
>>>> >
>>>> >
>>>> >Thanks,
>>>> >-Nitin
>>>> >
>>>> >On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
>>>> >
>>>> >>Hi Nitin,
>>>> >>
>>>> >>Please see below
>>>> >>
>>>> >>Hari
>>>> >>
>>>> >>-----Original Message-----
>>>> >>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
>>>> >>Sent: Tuesday, February 12, 2013 7:15 AM
>>>> >>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
>>>> >>Abhinandan Prateek
>>>> >>Cc: Chip Childers
>>>> >>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>> >>
>>>> >>Apologize for the delayed response. Was involved in other issues.
>>>> >>Please find answers inline.
>>>> >>
>>>> >>>
>>>> >>>-----Original Message-----
>>>> >>>From: Prashant Kumar Mishra
>>>>[mailto:prashantkumar.mishra@citrix.com]
>>>> >>>Sent: Thursday, January 24, 2013 12:26 PM
>>>> >>>To: cloudstack-dev@incubator.apache.org
>>>> >>>Cc: Nitin Mehta
>>>> >>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>> >>>
>>>> >>>Hi Nitin,
>>>> >>>I am planning to take the QA job for this feature. Have reviewed
>>>>the
>>>> >>>functional spec, gone through community discussion and have the
>>>> >>>following questions
>>>> >>>
>>>> >>>1-What is expected behavior of CS for Operating systems which do
>>>>not
>>>> >>>support dynamic scaling . ?
>>>> >>
>>>> >>Will throw a not supported exception
>>>> >>
>>>> >>Hari: How do we know which OS is supported or not? Is it going to be
>>>> >>part of the "capabilities" of hypervisor? Or where will this be
>>>> >>specified/configured?
>>>> >>PS: I know we plan to implement this only on VMware for now, but
>>>>when
>>>> >>installed/shipped, how will CS know the supported Hypervisor/OS?
>>>> >>
>>>> >>>
>>>> >>>2-How much resources can be scaled up, is it limited by
>>>>availability
>>>> >>>of resource on host .?
>>>> >>>
>>>> >>>[Koushik Das ]
>>>> >>>"Having a range for CPU/RAM in compute offering is definitely
>>>>another
>>>> >>>way of doing it. But creating the higher limit would be tricky. I
>>>>am
>>>> >>>not sure if it is always known to users how much they want to scale
>>>> >>>up to at the time of deploying VM. Moreover if the higher limit is
>>>> >>>known then the VM can be deployed with that value itself. Also in
>>>> >>>case of having a range in the offering the usage part needs to be
>>>> >>>handled appropriately. Currently usage is purely based on the
>>>> >>>offering and individual values are not stored".
>>>> >>>[/Koushik Das]
>>>> >>
>>>> >>It is not limited by the resources on host but on the available
>>>> >>capacity in any of the host within the cluster.
>>>> >>
>>>> >>Hari: I'm not sure I understand the question - how is this any
>>>> >>different than requesting a new VM? Or upgrading from offering A to
>>>> >>Offering B that exists today (although VM needs to be shutdown)?
>>>> >>
>>>> >>>
>>>> >>>it seems its totally depend on service offering , please correct
>>>>me
>>>> >>>if I am wrong.
>>>> >>>
>>>> >>>3- Scheduled snapshot of volumes during the operation .
>>>> >>>
>>>> >>>[NITIN]
>>>> >>>For vmware, the entire vm is locked by HV and this can be an issue.
>>>>I
>>>> >>>will leverage on current implementations for existing interactions
>>>> >>>like scheduled snapshots events during live migration and will
>>>> >>>replicate the same.
>>>> >>>[/NITIN]
>>>> >>>
>>>> >>>Can you elaborate what is expected in case of VMware .
>>>> >>
>>>> >>What I mean is there is an existing functionality which is
>>>>implemented
>>>> >>the same way. I will just do it the same way.
>>>> >>
>>>> >>>
>>>> >>>4 - what is expected behavior in case of powers off the vm during
>>>> >>>the operation .? is it different for different hypervisors.?
>>>> >>
>>>> >>There is nothing much to do for powered off vms because we will just
>>>> >>update the DB. When the vm is started it will pick up these values
>>>> >>from the DB.
>>>> >>This functionality already exists.
>>>> >>
>>>> >>>
>>>> >>>5- what is expected in case of migration fails( In FS no
>>>>description
>>>> >>>about this),
>>>> >>> -CS will retry to migrate it again if yes how many time ?
>>>> >>> - will it mark as a failure and can't scale up(even
>>>>resources
>>>> >>>are available in cluster) ?
>>>> >>
>>>> >>We will retry N (configurable) times and if unsuccessful we will
>>>>throw
>>>> >>an exception to the user.
>>>> >>
>>>> >>Hari: Can you please elaborate why a migration might fail? And, is
>>>>the
>>>> >>"N" configurable times a new global?
>>>> >>
>>>> >>>
>>>> >>>6- Apart from "scaleVirtualMachine" any other APIs are getting
>>>> >>>changed ?
>>>> >>
>>>> >>No
>>>> >>
>>>> >>>
>>>> >>>7-Scale down is allowed ? (still open issue in FS)
>>>> >>
>>>> >>No for time being.
>>>> >>
>>>> >>>
>>>> >>>8-Are we going to introduce custom compute offering (still open
>>>>issue
>>>> >>>in
>>>> >>>FS) ?
>>>> >>
>>>> >>No for now
>>>> >>
>>>> >>>
>>>> >>>9- what are the guide line for upgrade ?
>>>> >>
>>>> >>There is nothing for upgrade because we do not introduce new values
>>>>in
>>>> >>DB.
>>>> >>
>>>> >>>
>>>> >>>10-Any DB changes ?
>>>> >>
>>>> >>See #9 above.
>>>> >>
>>>> >>>
>>>> >>>11- which Usage events are getting introduced for billing .?
>>>> >>
>>>> >>Will update the FS.
>>>> >>
>>>> >>>
>>>> >>>12-hypervisor support ,is it only for VMware (as per FS) or its
>>>> >>>getting extended for XS/KVM also ?
>>>> >>
>>>> >>There are subtasks opened for XS and KVM. I am doing it only for
>>>>Vmware.
>>>> >>
>>>> >>>
>>>> >>>Thanks
>>>> >>>Prashant Kumar Mishra
>>>> >>>
>>>> >>>
>>>> >>>-----Original Message-----
>>>> >>>From: Koushik Das [mailto:koushik.das@citrix.com]
>>>> >>>Sent: Saturday, December 15, 2012 11:14 PM
>>>> >>>To: cloudstack-dev@incubator.apache.org
>>>> >>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>> >>>
>>>> >>>Currently CS supports changing CPU and RAM for stopped VM. This is
>>>> >>>achieved by changing compute offering of the VM (with new CPU and
>>>> RAM
>>>> >>>values) and then starting it. I am planning to extend the same for
>>>> >>>running VM as well. Initially planning to do it for Vmware where
>>>>CPU
>>>> >>>and RAM can be dynamically increased. Support of other HVs can also
>>>> >>>be added if they support increasing CPU/RAM.
>>>> >>>
>>>> >>>Assuming that in the updated compute offering only CPU and RAM has
>>>> >>>changed, the deployment planner can either select the same host in
>>>> >>>which case the values are dynamically scaled up OR a different one
>>>>in
>>>> >>>which case the operation fails. In future if there is support for
>>>> >>>live migration (provided HV supports it) then another option in the
>>>> >>>latter case could be to migrate the VM first and then scale it up.
>>>> >>>
>>>> >>>I will start working on the FS and share it out sometime next week.
>>>> >>>
>>>> >>>Comments/suggestions?
>>>> >>>
>>>> >>>Thanks,
>>>> >>>Koushik
>>>> >>
>>>> >
>>>
>>
>
Re: [DISCUSS] Scaling up CPU and RAM for running VMs
Posted by Kelven Yang <ke...@citrix.com>.
Nitin,
Sorry to reply late, have been busy working on a patch release for
customer.
AsyncJob manager does provide a mechanism that you can do synchronized the
job execution against a object. You may check out
AsyncJobManager.syncAsyncJobExecution().
Kelven
On 3/12/13 11:27 AM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>syncAsyncJobExecutionThanks Alex.
>
>Kelven / Alex - I took a look into this and it seems that the framework
>has the capability to do this and this would solve the problem for scaling
>up vm and vm snapshots.
>But, we currently don't do synchronization for the vm object. Doing this
>means all the vm operations will be syncronized now. Are we fine with that
>?
>
>Thanks,
>-Nitin
>
>On 09/03/13 10:36 PM, "Alex Huang" <Al...@citrix.com> wrote:
>
>>Nitin,
>>
>>The other approach to this is to utilize the syncing feature in the job
>>queue. I've cced Kelven to see if he can give you more detail. His code
>>is capable of syncing operations on a single object so you don't have to
>>add processing states.
>>
>>Given that all of your operations and vm snapshot operations must have
>>come in through the job queue, you might already have that ability to not
>>interfere with each other.
>>
>>VM States are different because there can be outside changes (through
>>other vm managers) that cause vm life cycle to behave differently.
>>
>>--Alex
>>
>>> -----Original Message-----
>>> From: Nitin Mehta
>>> Sent: Saturday, March 9, 2013 2:35 AM
>>> To: cloudstack-dev@incubator.apache.org; Prashant Kumar Mishra;
>>> Abhinandan Prateek; Alex Huang
>>> Cc: Chip Childers
>>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>
>>> Hi Alex,
>>> I had one more question. Say the MS is shut down or restarted, when do
>>>we
>>> clear this attribute in this case ?
>>>
>>> On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>>>
>>> >Alex - Thanks very much for pointing out earlier this week that for
>>> >scaling up the vm we shouldn't change the vm lifecycle. I also read
>>> >http://markmail.org/message/6c6njactsklot62h
>>> >and understand that scaling up a vm is a vm operation and shouldn't be
>>> >mixed with vm lifecycle. So as you suggested in the thread that if I
>>> >need to prevent other Vm operations happening during this operation I
>>> >would need to introduce an attribute
>>> >
>>> >1. For this I would need to introduce a column in vm_instance table
>>> >which would be set during scale up operation.
>>> >2. To prevent other operations from happening this attribute needs to
>>> >be checked in all the other vm operations. There is no single common
>>> >piece of code where I can put the check so I have to explicitly check
>>> >for this attribute in all the operations code right ? I see that for
>>>"vm
>>> snapshot"
>>> >operation we have put this check in vm state transition method but
>>>this
>>> >method is called only for vm lifecycle changes. So when "vm snapshot"
>>> >happens the user might also scale up the vm. There might be a need for
>>> >them to be exclusive.
>>> >3. If I need to say lock capacity before the operation and modify it
>>> >after the operation is done (say during failure) how do I do it w/o
>>> >coupling the code changes or is it ok for now to do so ?
>>> >
>>> >
>>> >Thanks,
>>> >-Nitin
>>> >
>>> >On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
>>> >
>>> >>Hi Nitin,
>>> >>
>>> >>Please see below
>>> >>
>>> >>Hari
>>> >>
>>> >>-----Original Message-----
>>> >>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
>>> >>Sent: Tuesday, February 12, 2013 7:15 AM
>>> >>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
>>> >>Abhinandan Prateek
>>> >>Cc: Chip Childers
>>> >>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> >>
>>> >>Apologize for the delayed response. Was involved in other issues.
>>> >>Please find answers inline.
>>> >>
>>> >>>
>>> >>>-----Original Message-----
>>> >>>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
>>> >>>Sent: Thursday, January 24, 2013 12:26 PM
>>> >>>To: cloudstack-dev@incubator.apache.org
>>> >>>Cc: Nitin Mehta
>>> >>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> >>>
>>> >>>Hi Nitin,
>>> >>>I am planning to take the QA job for this feature. Have reviewed the
>>> >>>functional spec, gone through community discussion and have the
>>> >>>following questions
>>> >>>
>>> >>>1-What is expected behavior of CS for Operating systems which do not
>>> >>>support dynamic scaling . ?
>>> >>
>>> >>Will throw a not supported exception
>>> >>
>>> >>Hari: How do we know which OS is supported or not? Is it going to be
>>> >>part of the "capabilities" of hypervisor? Or where will this be
>>> >>specified/configured?
>>> >>PS: I know we plan to implement this only on VMware for now, but when
>>> >>installed/shipped, how will CS know the supported Hypervisor/OS?
>>> >>
>>> >>>
>>> >>>2-How much resources can be scaled up, is it limited by availability
>>> >>>of resource on host .?
>>> >>>
>>> >>>[Koushik Das ]
>>> >>>"Having a range for CPU/RAM in compute offering is definitely
>>>another
>>> >>>way of doing it. But creating the higher limit would be tricky. I am
>>> >>>not sure if it is always known to users how much they want to scale
>>> >>>up to at the time of deploying VM. Moreover if the higher limit is
>>> >>>known then the VM can be deployed with that value itself. Also in
>>> >>>case of having a range in the offering the usage part needs to be
>>> >>>handled appropriately. Currently usage is purely based on the
>>> >>>offering and individual values are not stored".
>>> >>>[/Koushik Das]
>>> >>
>>> >>It is not limited by the resources on host but on the available
>>> >>capacity in any of the host within the cluster.
>>> >>
>>> >>Hari: I'm not sure I understand the question - how is this any
>>> >>different than requesting a new VM? Or upgrading from offering A to
>>> >>Offering B that exists today (although VM needs to be shutdown)?
>>> >>
>>> >>>
>>> >>>it seems its totally depend on service offering , please correct me
>>> >>>if I am wrong.
>>> >>>
>>> >>>3- Scheduled snapshot of volumes during the operation .
>>> >>>
>>> >>>[NITIN]
>>> >>>For vmware, the entire vm is locked by HV and this can be an issue.
>>>I
>>> >>>will leverage on current implementations for existing interactions
>>> >>>like scheduled snapshots events during live migration and will
>>> >>>replicate the same.
>>> >>>[/NITIN]
>>> >>>
>>> >>>Can you elaborate what is expected in case of VMware .
>>> >>
>>> >>What I mean is there is an existing functionality which is
>>>implemented
>>> >>the same way. I will just do it the same way.
>>> >>
>>> >>>
>>> >>>4 - what is expected behavior in case of powers off the vm during
>>> >>>the operation .? is it different for different hypervisors.?
>>> >>
>>> >>There is nothing much to do for powered off vms because we will just
>>> >>update the DB. When the vm is started it will pick up these values
>>> >>from the DB.
>>> >>This functionality already exists.
>>> >>
>>> >>>
>>> >>>5- what is expected in case of migration fails( In FS no description
>>> >>>about this),
>>> >>> -CS will retry to migrate it again if yes how many time ?
>>> >>> - will it mark as a failure and can't scale up(even resources
>>> >>>are available in cluster) ?
>>> >>
>>> >>We will retry N (configurable) times and if unsuccessful we will
>>>throw
>>> >>an exception to the user.
>>> >>
>>> >>Hari: Can you please elaborate why a migration might fail? And, is
>>>the
>>> >>"N" configurable times a new global?
>>> >>
>>> >>>
>>> >>>6- Apart from "scaleVirtualMachine" any other APIs are getting
>>> >>>changed ?
>>> >>
>>> >>No
>>> >>
>>> >>>
>>> >>>7-Scale down is allowed ? (still open issue in FS)
>>> >>
>>> >>No for time being.
>>> >>
>>> >>>
>>> >>>8-Are we going to introduce custom compute offering (still open
>>>issue
>>> >>>in
>>> >>>FS) ?
>>> >>
>>> >>No for now
>>> >>
>>> >>>
>>> >>>9- what are the guide line for upgrade ?
>>> >>
>>> >>There is nothing for upgrade because we do not introduce new values
>>>in
>>> >>DB.
>>> >>
>>> >>>
>>> >>>10-Any DB changes ?
>>> >>
>>> >>See #9 above.
>>> >>
>>> >>>
>>> >>>11- which Usage events are getting introduced for billing .?
>>> >>
>>> >>Will update the FS.
>>> >>
>>> >>>
>>> >>>12-hypervisor support ,is it only for VMware (as per FS) or its
>>> >>>getting extended for XS/KVM also ?
>>> >>
>>> >>There are subtasks opened for XS and KVM. I am doing it only for
>>>Vmware.
>>> >>
>>> >>>
>>> >>>Thanks
>>> >>>Prashant Kumar Mishra
>>> >>>
>>> >>>
>>> >>>-----Original Message-----
>>> >>>From: Koushik Das [mailto:koushik.das@citrix.com]
>>> >>>Sent: Saturday, December 15, 2012 11:14 PM
>>> >>>To: cloudstack-dev@incubator.apache.org
>>> >>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> >>>
>>> >>>Currently CS supports changing CPU and RAM for stopped VM. This is
>>> >>>achieved by changing compute offering of the VM (with new CPU and
>>> RAM
>>> >>>values) and then starting it. I am planning to extend the same for
>>> >>>running VM as well. Initially planning to do it for Vmware where CPU
>>> >>>and RAM can be dynamically increased. Support of other HVs can also
>>> >>>be added if they support increasing CPU/RAM.
>>> >>>
>>> >>>Assuming that in the updated compute offering only CPU and RAM has
>>> >>>changed, the deployment planner can either select the same host in
>>> >>>which case the values are dynamically scaled up OR a different one
>>>in
>>> >>>which case the operation fails. In future if there is support for
>>> >>>live migration (provided HV supports it) then another option in the
>>> >>>latter case could be to migrate the VM first and then scale it up.
>>> >>>
>>> >>>I will start working on the FS and share it out sometime next week.
>>> >>>
>>> >>>Comments/suggestions?
>>> >>>
>>> >>>Thanks,
>>> >>>Koushik
>>> >>
>>> >
>>
>
Re: [DISCUSS] Scaling up CPU and RAM for running VMs
Posted by Nitin Mehta <Ni...@citrix.com>.
Thanks Alex.
Kelven / Alex - I took a look into this and it seems that the framework
has the capability to do this and this would solve the problem for scaling
up vm and vm snapshots.
But, we currently don't do synchronization for the vm object. Doing this
means all the vm operations will be syncronized now. Are we fine with that
?
Thanks,
-Nitin
On 09/03/13 10:36 PM, "Alex Huang" <Al...@citrix.com> wrote:
>Nitin,
>
>The other approach to this is to utilize the syncing feature in the job
>queue. I've cced Kelven to see if he can give you more detail. His code
>is capable of syncing operations on a single object so you don't have to
>add processing states.
>
>Given that all of your operations and vm snapshot operations must have
>come in through the job queue, you might already have that ability to not
>interfere with each other.
>
>VM States are different because there can be outside changes (through
>other vm managers) that cause vm life cycle to behave differently.
>
>--Alex
>
>> -----Original Message-----
>> From: Nitin Mehta
>> Sent: Saturday, March 9, 2013 2:35 AM
>> To: cloudstack-dev@incubator.apache.org; Prashant Kumar Mishra;
>> Abhinandan Prateek; Alex Huang
>> Cc: Chip Childers
>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>
>> Hi Alex,
>> I had one more question. Say the MS is shut down or restarted, when do
>>we
>> clear this attribute in this case ?
>>
>> On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>>
>> >Alex - Thanks very much for pointing out earlier this week that for
>> >scaling up the vm we shouldn't change the vm lifecycle. I also read
>> >http://markmail.org/message/6c6njactsklot62h
>> >and understand that scaling up a vm is a vm operation and shouldn't be
>> >mixed with vm lifecycle. So as you suggested in the thread that if I
>> >need to prevent other Vm operations happening during this operation I
>> >would need to introduce an attribute
>> >
>> >1. For this I would need to introduce a column in vm_instance table
>> >which would be set during scale up operation.
>> >2. To prevent other operations from happening this attribute needs to
>> >be checked in all the other vm operations. There is no single common
>> >piece of code where I can put the check so I have to explicitly check
>> >for this attribute in all the operations code right ? I see that for
>>"vm
>> snapshot"
>> >operation we have put this check in vm state transition method but this
>> >method is called only for vm lifecycle changes. So when "vm snapshot"
>> >happens the user might also scale up the vm. There might be a need for
>> >them to be exclusive.
>> >3. If I need to say lock capacity before the operation and modify it
>> >after the operation is done (say during failure) how do I do it w/o
>> >coupling the code changes or is it ok for now to do so ?
>> >
>> >
>> >Thanks,
>> >-Nitin
>> >
>> >On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
>> >
>> >>Hi Nitin,
>> >>
>> >>Please see below
>> >>
>> >>Hari
>> >>
>> >>-----Original Message-----
>> >>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
>> >>Sent: Tuesday, February 12, 2013 7:15 AM
>> >>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
>> >>Abhinandan Prateek
>> >>Cc: Chip Childers
>> >>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>> >>
>> >>Apologize for the delayed response. Was involved in other issues.
>> >>Please find answers inline.
>> >>
>> >>>
>> >>>-----Original Message-----
>> >>>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
>> >>>Sent: Thursday, January 24, 2013 12:26 PM
>> >>>To: cloudstack-dev@incubator.apache.org
>> >>>Cc: Nitin Mehta
>> >>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>> >>>
>> >>>Hi Nitin,
>> >>>I am planning to take the QA job for this feature. Have reviewed the
>> >>>functional spec, gone through community discussion and have the
>> >>>following questions
>> >>>
>> >>>1-What is expected behavior of CS for Operating systems which do not
>> >>>support dynamic scaling . ?
>> >>
>> >>Will throw a not supported exception
>> >>
>> >>Hari: How do we know which OS is supported or not? Is it going to be
>> >>part of the "capabilities" of hypervisor? Or where will this be
>> >>specified/configured?
>> >>PS: I know we plan to implement this only on VMware for now, but when
>> >>installed/shipped, how will CS know the supported Hypervisor/OS?
>> >>
>> >>>
>> >>>2-How much resources can be scaled up, is it limited by availability
>> >>>of resource on host .?
>> >>>
>> >>>[Koushik Das ]
>> >>>"Having a range for CPU/RAM in compute offering is definitely another
>> >>>way of doing it. But creating the higher limit would be tricky. I am
>> >>>not sure if it is always known to users how much they want to scale
>> >>>up to at the time of deploying VM. Moreover if the higher limit is
>> >>>known then the VM can be deployed with that value itself. Also in
>> >>>case of having a range in the offering the usage part needs to be
>> >>>handled appropriately. Currently usage is purely based on the
>> >>>offering and individual values are not stored".
>> >>>[/Koushik Das]
>> >>
>> >>It is not limited by the resources on host but on the available
>> >>capacity in any of the host within the cluster.
>> >>
>> >>Hari: I'm not sure I understand the question - how is this any
>> >>different than requesting a new VM? Or upgrading from offering A to
>> >>Offering B that exists today (although VM needs to be shutdown)?
>> >>
>> >>>
>> >>>it seems its totally depend on service offering , please correct me
>> >>>if I am wrong.
>> >>>
>> >>>3- Scheduled snapshot of volumes during the operation .
>> >>>
>> >>>[NITIN]
>> >>>For vmware, the entire vm is locked by HV and this can be an issue. I
>> >>>will leverage on current implementations for existing interactions
>> >>>like scheduled snapshots events during live migration and will
>> >>>replicate the same.
>> >>>[/NITIN]
>> >>>
>> >>>Can you elaborate what is expected in case of VMware .
>> >>
>> >>What I mean is there is an existing functionality which is implemented
>> >>the same way. I will just do it the same way.
>> >>
>> >>>
>> >>>4 - what is expected behavior in case of powers off the vm during
>> >>>the operation .? is it different for different hypervisors.?
>> >>
>> >>There is nothing much to do for powered off vms because we will just
>> >>update the DB. When the vm is started it will pick up these values
>> >>from the DB.
>> >>This functionality already exists.
>> >>
>> >>>
>> >>>5- what is expected in case of migration fails( In FS no description
>> >>>about this),
>> >>> -CS will retry to migrate it again if yes how many time ?
>> >>> - will it mark as a failure and can't scale up(even resources
>> >>>are available in cluster) ?
>> >>
>> >>We will retry N (configurable) times and if unsuccessful we will throw
>> >>an exception to the user.
>> >>
>> >>Hari: Can you please elaborate why a migration might fail? And, is the
>> >>"N" configurable times a new global?
>> >>
>> >>>
>> >>>6- Apart from "scaleVirtualMachine" any other APIs are getting
>> >>>changed ?
>> >>
>> >>No
>> >>
>> >>>
>> >>>7-Scale down is allowed ? (still open issue in FS)
>> >>
>> >>No for time being.
>> >>
>> >>>
>> >>>8-Are we going to introduce custom compute offering (still open issue
>> >>>in
>> >>>FS) ?
>> >>
>> >>No for now
>> >>
>> >>>
>> >>>9- what are the guide line for upgrade ?
>> >>
>> >>There is nothing for upgrade because we do not introduce new values in
>> >>DB.
>> >>
>> >>>
>> >>>10-Any DB changes ?
>> >>
>> >>See #9 above.
>> >>
>> >>>
>> >>>11- which Usage events are getting introduced for billing .?
>> >>
>> >>Will update the FS.
>> >>
>> >>>
>> >>>12-hypervisor support ,is it only for VMware (as per FS) or its
>> >>>getting extended for XS/KVM also ?
>> >>
>> >>There are subtasks opened for XS and KVM. I am doing it only for
>>Vmware.
>> >>
>> >>>
>> >>>Thanks
>> >>>Prashant Kumar Mishra
>> >>>
>> >>>
>> >>>-----Original Message-----
>> >>>From: Koushik Das [mailto:koushik.das@citrix.com]
>> >>>Sent: Saturday, December 15, 2012 11:14 PM
>> >>>To: cloudstack-dev@incubator.apache.org
>> >>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>> >>>
>> >>>Currently CS supports changing CPU and RAM for stopped VM. This is
>> >>>achieved by changing compute offering of the VM (with new CPU and
>> RAM
>> >>>values) and then starting it. I am planning to extend the same for
>> >>>running VM as well. Initially planning to do it for Vmware where CPU
>> >>>and RAM can be dynamically increased. Support of other HVs can also
>> >>>be added if they support increasing CPU/RAM.
>> >>>
>> >>>Assuming that in the updated compute offering only CPU and RAM has
>> >>>changed, the deployment planner can either select the same host in
>> >>>which case the values are dynamically scaled up OR a different one in
>> >>>which case the operation fails. In future if there is support for
>> >>>live migration (provided HV supports it) then another option in the
>> >>>latter case could be to migrate the VM first and then scale it up.
>> >>>
>> >>>I will start working on the FS and share it out sometime next week.
>> >>>
>> >>>Comments/suggestions?
>> >>>
>> >>>Thanks,
>> >>>Koushik
>> >>
>> >
>
RE: [DISCUSS] Scaling up CPU and RAM for running VMs
Posted by Alex Huang <Al...@citrix.com>.
Nitin,
The other approach to this is to utilize the syncing feature in the job queue. I've cced Kelven to see if he can give you more detail. His code is capable of syncing operations on a single object so you don't have to add processing states.
Given that all of your operations and vm snapshot operations must have come in through the job queue, you might already have that ability to not interfere with each other.
VM States are different because there can be outside changes (through other vm managers) that cause vm life cycle to behave differently.
--Alex
> -----Original Message-----
> From: Nitin Mehta
> Sent: Saturday, March 9, 2013 2:35 AM
> To: cloudstack-dev@incubator.apache.org; Prashant Kumar Mishra;
> Abhinandan Prateek; Alex Huang
> Cc: Chip Childers
> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>
> Hi Alex,
> I had one more question. Say the MS is shut down or restarted, when do we
> clear this attribute in this case ?
>
> On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>
> >Alex - Thanks very much for pointing out earlier this week that for
> >scaling up the vm we shouldn't change the vm lifecycle. I also read
> >http://markmail.org/message/6c6njactsklot62h
> >and understand that scaling up a vm is a vm operation and shouldn't be
> >mixed with vm lifecycle. So as you suggested in the thread that if I
> >need to prevent other Vm operations happening during this operation I
> >would need to introduce an attribute
> >
> >1. For this I would need to introduce a column in vm_instance table
> >which would be set during scale up operation.
> >2. To prevent other operations from happening this attribute needs to
> >be checked in all the other vm operations. There is no single common
> >piece of code where I can put the check so I have to explicitly check
> >for this attribute in all the operations code right ? I see that for "vm
> snapshot"
> >operation we have put this check in vm state transition method but this
> >method is called only for vm lifecycle changes. So when "vm snapshot"
> >happens the user might also scale up the vm. There might be a need for
> >them to be exclusive.
> >3. If I need to say lock capacity before the operation and modify it
> >after the operation is done (say during failure) how do I do it w/o
> >coupling the code changes or is it ok for now to do so ?
> >
> >
> >Thanks,
> >-Nitin
> >
> >On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
> >
> >>Hi Nitin,
> >>
> >>Please see below
> >>
> >>Hari
> >>
> >>-----Original Message-----
> >>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
> >>Sent: Tuesday, February 12, 2013 7:15 AM
> >>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
> >>Abhinandan Prateek
> >>Cc: Chip Childers
> >>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>
> >>Apologize for the delayed response. Was involved in other issues.
> >>Please find answers inline.
> >>
> >>>
> >>>-----Original Message-----
> >>>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
> >>>Sent: Thursday, January 24, 2013 12:26 PM
> >>>To: cloudstack-dev@incubator.apache.org
> >>>Cc: Nitin Mehta
> >>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>
> >>>Hi Nitin,
> >>>I am planning to take the QA job for this feature. Have reviewed the
> >>>functional spec, gone through community discussion and have the
> >>>following questions
> >>>
> >>>1-What is expected behavior of CS for Operating systems which do not
> >>>support dynamic scaling . ?
> >>
> >>Will throw a not supported exception
> >>
> >>Hari: How do we know which OS is supported or not? Is it going to be
> >>part of the "capabilities" of hypervisor? Or where will this be
> >>specified/configured?
> >>PS: I know we plan to implement this only on VMware for now, but when
> >>installed/shipped, how will CS know the supported Hypervisor/OS?
> >>
> >>>
> >>>2-How much resources can be scaled up, is it limited by availability
> >>>of resource on host .?
> >>>
> >>>[Koushik Das ]
> >>>"Having a range for CPU/RAM in compute offering is definitely another
> >>>way of doing it. But creating the higher limit would be tricky. I am
> >>>not sure if it is always known to users how much they want to scale
> >>>up to at the time of deploying VM. Moreover if the higher limit is
> >>>known then the VM can be deployed with that value itself. Also in
> >>>case of having a range in the offering the usage part needs to be
> >>>handled appropriately. Currently usage is purely based on the
> >>>offering and individual values are not stored".
> >>>[/Koushik Das]
> >>
> >>It is not limited by the resources on host but on the available
> >>capacity in any of the host within the cluster.
> >>
> >>Hari: I'm not sure I understand the question - how is this any
> >>different than requesting a new VM? Or upgrading from offering A to
> >>Offering B that exists today (although VM needs to be shutdown)?
> >>
> >>>
> >>>it seems its totally depend on service offering , please correct me
> >>>if I am wrong.
> >>>
> >>>3- Scheduled snapshot of volumes during the operation .
> >>>
> >>>[NITIN]
> >>>For vmware, the entire vm is locked by HV and this can be an issue. I
> >>>will leverage on current implementations for existing interactions
> >>>like scheduled snapshots events during live migration and will
> >>>replicate the same.
> >>>[/NITIN]
> >>>
> >>>Can you elaborate what is expected in case of VMware .
> >>
> >>What I mean is there is an existing functionality which is implemented
> >>the same way. I will just do it the same way.
> >>
> >>>
> >>>4 - what is expected behavior in case of powers off the vm during
> >>>the operation .? is it different for different hypervisors.?
> >>
> >>There is nothing much to do for powered off vms because we will just
> >>update the DB. When the vm is started it will pick up these values
> >>from the DB.
> >>This functionality already exists.
> >>
> >>>
> >>>5- what is expected in case of migration fails( In FS no description
> >>>about this),
> >>> -CS will retry to migrate it again if yes how many time ?
> >>> - will it mark as a failure and can't scale up(even resources
> >>>are available in cluster) ?
> >>
> >>We will retry N (configurable) times and if unsuccessful we will throw
> >>an exception to the user.
> >>
> >>Hari: Can you please elaborate why a migration might fail? And, is the
> >>"N" configurable times a new global?
> >>
> >>>
> >>>6- Apart from "scaleVirtualMachine" any other APIs are getting
> >>>changed ?
> >>
> >>No
> >>
> >>>
> >>>7-Scale down is allowed ? (still open issue in FS)
> >>
> >>No for time being.
> >>
> >>>
> >>>8-Are we going to introduce custom compute offering (still open issue
> >>>in
> >>>FS) ?
> >>
> >>No for now
> >>
> >>>
> >>>9- what are the guide line for upgrade ?
> >>
> >>There is nothing for upgrade because we do not introduce new values in
> >>DB.
> >>
> >>>
> >>>10-Any DB changes ?
> >>
> >>See #9 above.
> >>
> >>>
> >>>11- which Usage events are getting introduced for billing .?
> >>
> >>Will update the FS.
> >>
> >>>
> >>>12-hypervisor support ,is it only for VMware (as per FS) or its
> >>>getting extended for XS/KVM also ?
> >>
> >>There are subtasks opened for XS and KVM. I am doing it only for Vmware.
> >>
> >>>
> >>>Thanks
> >>>Prashant Kumar Mishra
> >>>
> >>>
> >>>-----Original Message-----
> >>>From: Koushik Das [mailto:koushik.das@citrix.com]
> >>>Sent: Saturday, December 15, 2012 11:14 PM
> >>>To: cloudstack-dev@incubator.apache.org
> >>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>
> >>>Currently CS supports changing CPU and RAM for stopped VM. This is
> >>>achieved by changing compute offering of the VM (with new CPU and
> RAM
> >>>values) and then starting it. I am planning to extend the same for
> >>>running VM as well. Initially planning to do it for Vmware where CPU
> >>>and RAM can be dynamically increased. Support of other HVs can also
> >>>be added if they support increasing CPU/RAM.
> >>>
> >>>Assuming that in the updated compute offering only CPU and RAM has
> >>>changed, the deployment planner can either select the same host in
> >>>which case the values are dynamically scaled up OR a different one in
> >>>which case the operation fails. In future if there is support for
> >>>live migration (provided HV supports it) then another option in the
> >>>latter case could be to migrate the VM first and then scale it up.
> >>>
> >>>I will start working on the FS and share it out sometime next week.
> >>>
> >>>Comments/suggestions?
> >>>
> >>>Thanks,
> >>>Koushik
> >>
> >
Re: [DISCUSS] Scaling up CPU and RAM for running VMs
Posted by Nitin Mehta <Ni...@citrix.com>.
Hi Alex,
I had one more question. Say the MS is shut down or restarted, when do we
clear this attribute in this case ?
On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>Alex - Thanks very much for pointing out earlier this week that for
>scaling up the vm we shouldn't change the vm lifecycle. I also read
>http://markmail.org/message/6c6njactsklot62h
>and understand that scaling up a vm is a vm operation and shouldn't be
>mixed with vm lifecycle. So as you suggested in the thread that if I need
>to prevent other
>Vm operations happening during this operation I would need to introduce an
>attribute
>
>1. For this I would need to introduce a column in vm_instance table which
>would be set during scale up operation.
>2. To prevent other operations from happening this attribute needs to be
>checked in all the other vm operations. There is no single common piece of
>code where I can put the check so I have to explicitly check for this
>attribute in all the operations code right ? I see that for "vm snapshot"
>operation we have put this check in vm state transition method but this
>method is called only for vm lifecycle changes. So when "vm snapshot"
>happens the user might also scale up the vm. There might be a need for
>them to be exclusive.
>3. If I need to say lock capacity before the operation and modify it after
>the operation is done (say during failure) how do I do it w/o coupling the
>code changes or is it ok for now to do so ?
>
>
>Thanks,
>-Nitin
>
>On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
>
>>Hi Nitin,
>>
>>Please see below
>>
>>Hari
>>
>>-----Original Message-----
>>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
>>Sent: Tuesday, February 12, 2013 7:15 AM
>>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
>>Abhinandan Prateek
>>Cc: Chip Childers
>>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>
>>Apologize for the delayed response. Was involved in other issues.
>>Please find answers inline.
>>
>>>
>>>-----Original Message-----
>>>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
>>>Sent: Thursday, January 24, 2013 12:26 PM
>>>To: cloudstack-dev@incubator.apache.org
>>>Cc: Nitin Mehta
>>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>
>>>Hi Nitin,
>>>I am planning to take the QA job for this feature. Have reviewed the
>>>functional spec, gone through community discussion and have the
>>>following questions
>>>
>>>1-What is expected behavior of CS for Operating systems which do not
>>>support dynamic scaling . ?
>>
>>Will throw a not supported exception
>>
>>Hari: How do we know which OS is supported or not? Is it going to be part
>>of the "capabilities" of hypervisor? Or where will this be
>>specified/configured?
>>PS: I know we plan to implement this only on VMware for now, but when
>>installed/shipped, how will CS know the supported Hypervisor/OS?
>>
>>>
>>>2-How much resources can be scaled up, is it limited by availability of
>>>resource on host .?
>>>
>>>[Koushik Das ]
>>>"Having a range for CPU/RAM in compute offering is definitely another
>>>way of doing it. But creating the higher limit would be tricky. I am
>>>not sure if it is always known to users how much they want to scale up
>>>to at the time of deploying VM. Moreover if the higher limit is known
>>>then the VM can be deployed with that value itself. Also in case of
>>>having a range in the offering the usage part needs to be handled
>>>appropriately. Currently usage is purely based on the offering and
>>>individual values are not stored".
>>>[/Koushik Das]
>>
>>It is not limited by the resources on host but on the available capacity
>>in any of the host within the cluster.
>>
>>Hari: I'm not sure I understand the question - how is this any different
>>than requesting a new VM? Or upgrading from offering A to Offering B that
>>exists today (although VM needs to be shutdown)?
>>
>>>
>>>it seems its totally depend on service offering , please correct me if
>>>I am wrong.
>>>
>>>3- Scheduled snapshot of volumes during the operation .
>>>
>>>[NITIN]
>>>For vmware, the entire vm is locked by HV and this can be an issue. I
>>>will leverage on current implementations for existing interactions like
>>>scheduled snapshots events during live migration and will replicate the
>>>same.
>>>[/NITIN]
>>>
>>>Can you elaborate what is expected in case of VMware .
>>
>>What I mean is there is an existing functionality which is implemented
>>the same way. I will just do it the same way.
>>
>>>
>>>4 - what is expected behavior in case of powers off the vm during the
>>>operation .? is it different for different hypervisors.?
>>
>>There is nothing much to do for powered off vms because we will just
>>update the DB. When the vm is started it will pick up these values from
>>the DB.
>>This functionality already exists.
>>
>>>
>>>5- what is expected in case of migration fails( In FS no description
>>>about this),
>>> -CS will retry to migrate it again if yes how many time ?
>>> - will it mark as a failure and can't scale up(even resources
>>>are available in cluster) ?
>>
>>We will retry N (configurable) times and if unsuccessful we will throw an
>>exception to the user.
>>
>>Hari: Can you please elaborate why a migration might fail? And, is the
>>"N" configurable times a new global?
>>
>>>
>>>6- Apart from "scaleVirtualMachine" any other APIs are getting
>>>changed ?
>>
>>No
>>
>>>
>>>7-Scale down is allowed ? (still open issue in FS)
>>
>>No for time being.
>>
>>>
>>>8-Are we going to introduce custom compute offering (still open issue
>>>in
>>>FS) ?
>>
>>No for now
>>
>>>
>>>9- what are the guide line for upgrade ?
>>
>>There is nothing for upgrade because we do not introduce new values in
>>DB.
>>
>>>
>>>10-Any DB changes ?
>>
>>See #9 above.
>>
>>>
>>>11- which Usage events are getting introduced for billing .?
>>
>>Will update the FS.
>>
>>>
>>>12-hypervisor support ,is it only for VMware (as per FS) or its
>>>getting extended for XS/KVM also ?
>>
>>There are subtasks opened for XS and KVM. I am doing it only for Vmware.
>>
>>>
>>>Thanks
>>>Prashant Kumar Mishra
>>>
>>>
>>>-----Original Message-----
>>>From: Koushik Das [mailto:koushik.das@citrix.com]
>>>Sent: Saturday, December 15, 2012 11:14 PM
>>>To: cloudstack-dev@incubator.apache.org
>>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>
>>>Currently CS supports changing CPU and RAM for stopped VM. This is
>>>achieved by changing compute offering of the VM (with new CPU and RAM
>>>values) and then starting it. I am planning to extend the same for
>>>running VM as well. Initially planning to do it for Vmware where CPU
>>>and RAM can be dynamically increased. Support of other HVs can also be
>>>added if they support increasing CPU/RAM.
>>>
>>>Assuming that in the updated compute offering only CPU and RAM has
>>>changed, the deployment planner can either select the same host in
>>>which case the values are dynamically scaled up OR a different one in
>>>which case the operation fails. In future if there is support for live
>>>migration (provided HV supports it) then another option in the latter
>>>case could be to migrate the VM first and then scale it up.
>>>
>>>I will start working on the FS and share it out sometime next week.
>>>
>>>Comments/suggestions?
>>>
>>>Thanks,
>>>Koushik
>>
>