Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Mice Xia <we...@gmail.com>]

Nitin & Kelven,

I haven't updated the codes with syncAsyncJobExecution, and the conclusion
is making all vm api synchronized? I'm ok with this change, currently, for
a single vm instance, few apis can work on it concurrently.

-Mice


On Fri, Mar 15, 2013 at 12:20 PM, Nitin Mehta <Ni...@citrix.com>wrote:

> Thanks Kelven.
> I saw that, but making this change would mean that all the vm API's -
> deployvm, start-stop, reboot, destroy vm etc.. will now be synchronized
> with this change.
>
> Hope you guys are ok with this ??
>
> Thanks,
> -Nitin
>
> On 13/03/13 11:31 PM, "Kelven Yang" <ke...@citrix.com> wrote:
>
> >Nitin,
> >
> >Sorry to reply late, have been busy working on a patch release for
> >customer.
> >
> >AsyncJob manager does provide a mechanism that you can do synchronized the
> >job execution against a object. You may check out
> >AsyncJobManager.syncAsyncJobExecution().
> >
> >Kelven
> >
> >On 3/12/13 11:27 AM, "Nitin Mehta" <Ni...@citrix.com> wrote:
> >>syncAsyncJobExecutionThanks Alex.
> >>
> >>Kelven / Alex - I took a look into this and it seems that the framework
> >>has the capability to do this and this would solve the problem for
> >>scaling
> >>up vm and vm snapshots.
> >>But, we currently don't do synchronization for the vm object. Doing this
> >>means all the vm operations will be syncronized now. Are we fine with
> >>that
> >>?
> >>
> >>Thanks,
> >>-Nitin
> >>
> >>On 09/03/13 10:36 PM, "Alex Huang" <Al...@citrix.com> wrote:
> >>
> >>>Nitin,
> >>>
> >>>The other approach to this is to utilize the syncing feature in the job
> >>>queue.  I've cced Kelven to see if he can give you more detail.  His
> >>>code
> >>>is capable of syncing operations on a single object so you don't have to
> >>>add processing states.
> >>>
> >>>Given that all of your operations and vm snapshot operations must have
> >>>come in through the job queue, you might already have that ability to
> >>>not
> >>>interfere with each other.
> >>>
> >>>VM States are different because there can be outside changes (through
> >>>other vm managers) that cause vm life cycle to behave differently.
> >>>
> >>>--Alex
> >>>
> >>>> -----Original Message-----
> >>>> From: Nitin Mehta
> >>>> Sent: Saturday, March 9, 2013 2:35 AM
> >>>> To: cloudstack-dev@incubator.apache.org; Prashant Kumar Mishra;
> >>>> Abhinandan Prateek; Alex Huang
> >>>> Cc: Chip Childers
> >>>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>>
> >>>> Hi Alex,
> >>>> I had one more question. Say the MS is shut down or restarted, when do
> >>>>we
> >>>> clear this attribute in this case ?
> >>>>
> >>>> On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
> >>>>
> >>>> >Alex - Thanks very much for pointing out earlier this week that for
> >>>> >scaling up the vm we shouldn't change the vm lifecycle. I also read
> >>>> >http://markmail.org/message/6c6njactsklot62h
> >>>> >and understand that scaling up a vm is a vm operation and shouldn't
> >>>>be
> >>>> >mixed with vm lifecycle. So as you suggested in the thread that if I
> >>>> >need to prevent other Vm operations happening during this operation I
> >>>> >would need to introduce an attribute
> >>>> >
> >>>> >1. For this I would need to introduce a column in vm_instance table
> >>>> >which would be set during scale up operation.
> >>>> >2. To prevent other operations from happening this attribute needs to
> >>>> >be checked in all the other vm operations. There is no single common
> >>>> >piece of code where I can put the check so I have to explicitly check
> >>>> >for this attribute in all the operations code right ? I see that for
> >>>>"vm
> >>>> snapshot"
> >>>> >operation we have put this check in vm state transition method but
> >>>>this
> >>>> >method is called only for vm lifecycle changes. So when "vm snapshot"
> >>>> >happens the user might also scale up the vm. There might be a need
> >>>>for
> >>>> >them to be exclusive.
> >>>> >3. If I need to say lock capacity before the operation and modify it
> >>>> >after the operation is done (say during failure) how do I do it w/o
> >>>> >coupling the code changes or is it ok for now to do so ?
> >>>> >
> >>>> >
> >>>> >Thanks,
> >>>> >-Nitin
> >>>> >
> >>>> >On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
> >>>> >
> >>>> >>Hi Nitin,
> >>>> >>
> >>>> >>Please see below
> >>>> >>
> >>>> >>Hari
> >>>> >>
> >>>> >>-----Original Message-----
> >>>> >>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
> >>>> >>Sent: Tuesday, February 12, 2013 7:15 AM
> >>>> >>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
> >>>> >>Abhinandan Prateek
> >>>> >>Cc: Chip Childers
> >>>> >>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>> >>
> >>>> >>Apologize for the delayed response. Was involved in other issues.
> >>>> >>Please find answers inline.
> >>>> >>
> >>>> >>>
> >>>> >>>-----Original Message-----
> >>>> >>>From: Prashant Kumar Mishra
> >>>>[mailto:prashantkumar.mishra@citrix.com]
> >>>> >>>Sent: Thursday, January 24, 2013 12:26 PM
> >>>> >>>To: cloudstack-dev@incubator.apache.org
> >>>> >>>Cc: Nitin Mehta
> >>>> >>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>> >>>
> >>>> >>>Hi Nitin,
> >>>> >>>I am planning to take the QA job for this feature. Have reviewed
> >>>>the
> >>>> >>>functional spec, gone through community discussion  and have the
> >>>> >>>following questions
> >>>> >>>
> >>>> >>>1-What is expected behavior of CS for Operating systems which do
> >>>>not
> >>>> >>>support dynamic scaling . ?
> >>>> >>
> >>>> >>Will throw a not supported exception
> >>>> >>
> >>>> >>Hari: How do we know which OS is supported or not? Is it going to be
> >>>> >>part of the "capabilities" of hypervisor? Or where will this be
> >>>> >>specified/configured?
> >>>> >>PS: I know we plan to implement this only on VMware for now, but
> >>>>when
> >>>> >>installed/shipped, how will CS know the supported Hypervisor/OS?
> >>>> >>
> >>>> >>>
> >>>> >>>2-How much resources can be scaled up, is it limited by
> >>>>availability
> >>>> >>>of resource on host .?
> >>>> >>>
> >>>> >>>[Koushik Das ]
> >>>> >>>"Having a range for CPU/RAM in compute offering is definitely
> >>>>another
> >>>> >>>way of doing it. But creating the higher limit would be tricky. I
> >>>>am
> >>>> >>>not sure if it is always known to users how much they want to scale
> >>>> >>>up to at the time of deploying VM. Moreover if the higher limit is
> >>>> >>>known then the VM can be deployed with that value itself. Also in
> >>>> >>>case of having a range in the offering the usage part needs to be
> >>>> >>>handled appropriately. Currently usage is purely based on the
> >>>> >>>offering and individual values are not stored".
> >>>> >>>[/Koushik Das]
> >>>> >>
> >>>> >>It is not limited by the resources on host but on the available
> >>>> >>capacity in any of the host within the cluster.
> >>>> >>
> >>>> >>Hari: I'm not sure I understand the question - how is this any
> >>>> >>different than requesting a new VM? Or upgrading from offering A to
> >>>> >>Offering B that exists today (although VM needs to be shutdown)?
> >>>> >>
> >>>> >>>
> >>>> >>>it seems  its totally depend on service offering , please correct
> >>>>me
> >>>> >>>if I am wrong.
> >>>> >>>
> >>>> >>>3-  Scheduled snapshot of volumes during the operation .
> >>>> >>>
> >>>> >>>[NITIN]
> >>>> >>>For vmware, the entire vm is locked by HV and this can be an issue.
> >>>>I
> >>>> >>>will leverage on current implementations for existing interactions
> >>>> >>>like scheduled snapshots events during live migration and will
> >>>> >>>replicate the same.
> >>>> >>>[/NITIN]
> >>>> >>>
> >>>> >>>Can you elaborate what is expected in case of VMware .
> >>>> >>
> >>>> >>What I mean is there is an existing functionality which is
> >>>>implemented
> >>>> >>the same way. I will just do it the same way.
> >>>> >>
> >>>> >>>
> >>>> >>>4 - what is expected behavior in case of  powers off the vm during
> >>>> >>>the operation .? is it different for different hypervisors.?
> >>>> >>
> >>>> >>There is nothing much to do for powered off vms because we will just
> >>>> >>update the DB. When the vm is started it will pick up these values
> >>>> >>from the DB.
> >>>> >>This functionality already exists.
> >>>> >>
> >>>> >>>
> >>>> >>>5- what is expected in case of migration fails( In FS no
> >>>>description
> >>>> >>>about this),
> >>>> >>>       -CS will  retry to migrate it again if yes how many time ?
> >>>> >>>      - will it mark as a failure and can't  scale up(even
> >>>>resources
> >>>> >>>are available in cluster) ?
> >>>> >>
> >>>> >>We will retry N (configurable) times and if unsuccessful we will
> >>>>throw
> >>>> >>an exception to the user.
> >>>> >>
> >>>> >>Hari: Can you please elaborate why a migration might fail? And, is
> >>>>the
> >>>> >>"N" configurable times a new global?
> >>>> >>
> >>>> >>>
> >>>> >>>6- Apart from  "scaleVirtualMachine"    any other APIs are getting
> >>>> >>>changed ?
> >>>> >>
> >>>> >>No
> >>>> >>
> >>>> >>>
> >>>> >>>7-Scale down is allowed ? (still open issue in FS)
> >>>> >>
> >>>> >>No for time being.
> >>>> >>
> >>>> >>>
> >>>> >>>8-Are we going to introduce custom compute offering (still open
> >>>>issue
> >>>> >>>in
> >>>> >>>FS) ?
> >>>> >>
> >>>> >>No for now
> >>>> >>
> >>>> >>>
> >>>> >>>9- what are the guide line for upgrade  ?
> >>>> >>
> >>>> >>There is nothing for upgrade because we do not introduce new values
> >>>>in
> >>>> >>DB.
> >>>> >>
> >>>> >>>
> >>>> >>>10-Any DB changes ?
> >>>> >>
> >>>> >>See #9 above.
> >>>> >>
> >>>> >>>
> >>>> >>>11- which Usage events are getting introduced for billing .?
> >>>> >>
> >>>> >>Will update the FS.
> >>>> >>
> >>>> >>>
> >>>> >>>12-hypervisor support ,is it only for VMware (as per FS)  or its
> >>>> >>>getting extended for XS/KVM also ?
> >>>> >>
> >>>> >>There are subtasks opened for XS and KVM. I am doing it only for
> >>>>Vmware.
> >>>> >>
> >>>> >>>
> >>>> >>>Thanks
> >>>> >>>Prashant Kumar Mishra
> >>>> >>>
> >>>> >>>
> >>>> >>>-----Original Message-----
> >>>> >>>From: Koushik Das [mailto:koushik.das@citrix.com]
> >>>> >>>Sent: Saturday, December 15, 2012 11:14 PM
> >>>> >>>To: cloudstack-dev@incubator.apache.org
> >>>> >>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>> >>>
> >>>> >>>Currently CS supports changing CPU and RAM for stopped VM. This is
> >>>> >>>achieved by changing compute offering of the VM (with new CPU and
> >>>> RAM
> >>>> >>>values) and then starting it. I am planning to extend the same for
> >>>> >>>running VM as well. Initially planning to do it for Vmware where
> >>>>CPU
> >>>> >>>and RAM can be dynamically increased. Support of other HVs can also
> >>>> >>>be added if they support increasing CPU/RAM.
> >>>> >>>
> >>>> >>>Assuming that in the updated compute offering only CPU and RAM has
> >>>> >>>changed, the deployment planner can either select the same host in
> >>>> >>>which case the values are dynamically scaled up OR a different one
> >>>>in
> >>>> >>>which case the operation fails. In future if there is support for
> >>>> >>>live migration (provided HV supports it) then another option in the
> >>>> >>>latter case could be to migrate the VM first and then scale it up.
> >>>> >>>
> >>>> >>>I will start working on the FS and share it out sometime next week.
> >>>> >>>
> >>>> >>>Comments/suggestions?
> >>>> >>>
> >>>> >>>Thanks,
> >>>> >>>Koushik
> >>>> >>
> >>>> >
> >>>
> >>
> >
>
>