[DISCUSS] Scaling up CPU and RAM for running VMs

[Koushik Das <ko...@citrix.com>]

Currently CS supports changing CPU and RAM for stopped VM. This is achieved by changing compute offering of the VM (with new CPU and RAM values) and then starting it. I am planning to extend the same for running VM as well. Initially planning to do it for Vmware where CPU and RAM can be dynamically increased. Support of other HVs can also be added if they support increasing CPU/RAM.

Assuming that in the updated compute offering only CPU and RAM has changed, the deployment planner can either select the same host in which case the values are dynamically scaled up OR a different one in which case the operation fails. In future if there is support for live migration (provided HV supports it) then another option in the latter case could be to migrate the VM first and then scale it up.

I will start working on the FS and share it out sometime next week.

Comments/suggestions?

Thanks,
Koushik

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Nitin Mehta <Ni...@citrix.com>]

On 18-Dec-2012, at 8:54 AM, Mice Xia wrote:

> Great feature and I believe a lot of customers need it, hope Xenserver will support dynamic memory change one day.
> 
> [Koushik]
> Assuming that in the updated compute offering only CPU and RAM has changed, the deployment planner can either select the same host in which case the values are dynamically scaled up OR a different one in which case the operation fails. In future if there is support for live migration (provided HV supports it) then another option in the latter case could be to migrate the VM first and then scale it up.
> [/Koushik]
> 


Mice - Great point. But, I think when the vm is scaled up, allocators need to be wise enough to figure out if it could be adjusted on the same host or need live migration. I think atleast XS and Vmware support live migration (across clusters as well) now so dynamic scaling would have less probability of failing, but this could be an expensive operation because migrating volumes across clusters will be resource intensive.

> For VMware, it supports memory overprovisioning, which means the total size of VM's allocated memory can be larger than host's physical memory. Maybe we can leverage it and reduce the chance to migrate VM before scale up.
> 
> 
> Regards
> Mice
> 
> -----Original Message-----
> From: Koushik Das [mailto:koushik.das@citrix.com] 
> Sent: Sunday, December 16, 2012 1:44 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> 
> Currently CS supports changing CPU and RAM for stopped VM. This is achieved by changing compute offering of the VM (with new CPU and RAM values) and then starting it. I am planning to extend the same for running VM as well. Initially planning to do it for Vmware where CPU and RAM can be dynamically increased. Support of other HVs can also be added if they support increasing CPU/RAM.
> 
> Assuming that in the updated compute offering only CPU and RAM has changed, the deployment planner can either select the same host in which case the values are dynamically scaled up OR a different one in which case the operation fails. In future if there is support for live migration (provided HV supports it) then another option in the latter case could be to migrate the VM first and then scale it up.
> 
> I will start working on the FS and share it out sometime next week.
> 
> Comments/suggestions?
> 
> Thanks,
> Koushik

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Mice Xia <mi...@tcloudcomputing.com>]

Great feature and I believe a lot of customers need it, hope Xenserver will support dynamic memory change one day.

[Koushik]
Assuming that in the updated compute offering only CPU and RAM has changed, the deployment planner can either select the same host in which case the values are dynamically scaled up OR a different one in which case the operation fails. In future if there is support for live migration (provided HV supports it) then another option in the latter case could be to migrate the VM first and then scale it up.
[/Koushik]

For VMware, it supports memory overprovisioning, which means the total size of VM's allocated memory can be larger than host's physical memory. Maybe we can leverage it and reduce the chance to migrate VM before scale up.


Regards
Mice

-----Original Message-----
From: Koushik Das [mailto:koushik.das@citrix.com] 
Sent: Sunday, December 16, 2012 1:44 AM
To: cloudstack-dev@incubator.apache.org
Subject: [DISCUSS] Scaling up CPU and RAM for running VMs

Currently CS supports changing CPU and RAM for stopped VM. This is achieved by changing compute offering of the VM (with new CPU and RAM values) and then starting it. I am planning to extend the same for running VM as well. Initially planning to do it for Vmware where CPU and RAM can be dynamically increased. Support of other HVs can also be added if they support increasing CPU/RAM.

Assuming that in the updated compute offering only CPU and RAM has changed, the deployment planner can either select the same host in which case the values are dynamically scaled up OR a different one in which case the operation fails. In future if there is support for live migration (provided HV supports it) then another option in the latter case could be to migrate the VM first and then scale it up.

I will start working on the FS and share it out sometime next week.

Comments/suggestions?

Thanks,
Koushik

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

["Musayev, Ilya" <im...@webmd.net>]

Koushik

When you have something to test, let me know. We are big vmware shop, I can test this when needed.

FYI, I wont be able test anything until I get back in the offfice which the first week of January.

Last thing to note, not all operating systems support online increase of cpu/mem. Linux seems ok for the most part on 2.6.18+ kernels and 64bit, where windows has specific edition that can do that. Vmware has a doc on this somewhere.

Regards
Ilya

Koushik Das <ko...@citrix.com> wrote:
Currently CS supports changing CPU and RAM for stopped VM. This is achieved by changing compute offering of the VM (with new CPU and RAM values) and then starting it. I am planning to extend the same for running VM as well. Initially planning to do it for Vmware where CPU and RAM can be dynamically increased. Support of other HVs can also be added if they support increasing CPU/RAM.

Assuming that in the updated compute offering only CPU and RAM has changed, the deployment planner can either select the same host in which case the values are dynamically scaled up OR a different one in which case the operation fails. In future if there is support for live migration (provided HV supports it) then another option in the latter case could be to migrate the VM first and then scale it up.

I will start working on the FS and share it out sometime next week.

Comments/suggestions?

Thanks,
Koushik

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Marcus Sorensen <sh...@gmail.com>]

I was planning on looking at the same. With KVM, it requires that you
launch the vm with a max number and a current number. Then you can increase
up to max.

I agree that it will be important to migrate before scaling if necessary.
The scenario would likely be common and make the feature less useful if it
couldn't find resources in the cluster to fill the request. Perhaps there
could be various options such as "migrate to host with available resources"
or "migrate other vms to fufill request, smallest/largest first", or "least
cpu usage first" etc. That way if someone has a VM that needs to scale up
CPU it's not taking a performance penalty or outage by migrating, if the
admin chooses.


On Dec 15, 2012 10:45 AM, "Koushik Das" <ko...@citrix.com> wrote:

> Currently CS supports changing CPU and RAM for stopped VM. This is
> achieved by changing compute offering of the VM (with new CPU and RAM
> values) and then starting it. I am planning to extend the same for running
> VM as well. Initially planning to do it for Vmware where CPU and RAM can be
> dynamically increased. Support of other HVs can also be added if they
> support increasing CPU/RAM.
>
> Assuming that in the updated compute offering only CPU and RAM has
> changed, the deployment planner can either select the same host in which
> case the values are dynamically scaled up OR a different one in which case
> the operation fails. In future if there is support for live migration
> (provided HV supports it) then another option in the latter case could be
> to migrate the VM first and then scale it up.
>
> I will start working on the FS and share it out sometime next week.
>
> Comments/suggestions?
>
> Thanks,
> Koushik
>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Chiradeep Vittal <Ch...@citrix.com>]

You should also consider the case that the management server that has
initiated the scale-up fails in the middle of the procedure.

In general I would avoid locks as much as possible unless there is a
danger of corrupting the database. If for example the capacity calculation
says there is enough capacity for the scale-up operation, but when
actually implemented, there isn't, we can return a failure for the async
job.

Failures at the resource layer also raise the question: are we going to
retry? If so how many times? Which failures will require a retry, and
which ones will cause an abort?

I've also thought about the API design (changing it from synchronous to
asynchronous). I believe this is a big semantic change. I would introduce
a new API for this behavior.

On 1/22/13 5:54 AM, "Nitin Mehta" <Ni...@citrix.com> wrote:

>Chiradeep - thanks for your questions. Please find my answers inline. I
>might not have the best solutions for some of them so looking for some
>guidance as well.
>All these interactions make for good test cases so the QA for this feature
>should test these interactions.
>
>
>On 22/01/13 12:18 PM, "Chiradeep Vittal" <Ch...@citrix.com>
>wrote:
>
>>What usage events will be generated to ensure that the proper billing is
>>done?
>
>Currently usage implementation won't be able to handle this. I will
>introduce a
>new event to handle it for dynamically as well as statically(for Hvs that
>do not support this)
>
>>We need more details about the actual classes being affected and the
>>layers at which the orchestration is being done.
>
>I will put that information in the FS. It will be based on the flowchart
>in the FS.
>
>>What are the possible interactions that you are taking care of?
>> - user powers off the vm during the operation
>
>Need to explore how the HV is handling it. The HV should be handling it
>gracefully else its a bug on the HV.
>If the HV handles it cleanly then CS can accordingly surface it and take
>corrective actions like updating the DB etc.
>
>> - race conditions during calculation of sufficient capacity
>
>It will be implemented parallel to how we do it in allocators. As soon as
>we find a suitable destination we lock the capacity and release it in case
>of
>failures. All this is through state machine transitions so will be done
>cleanly.
>
>> - failure during live migration
>
>As you see in the flowchart in the FS I won't be touching the live
>migration but leveraging on the current implementation.
>I am assuming that we already handle it gracefully. But will definitely
>test it.
>
>> - HA event during live migration / upgrade
>
>I am assuming that we already handle the HA event during live migrations
>since this scenario is available in current implementation.
>Will try and leverage on the same for HA event during vm upgrade.
>
>> - scheduled snapshot of volumes during the operation
>
>For vmware, the entire vm is locked by HV and this can be an issue. I will
>leverage on current implementations for existing interactions like
>scheduled snapshots events during live migration and will replicate the
>same.
>
>
>> - attach / detach during the operation
>
>Same thing as the above.
>
>> - hypervisor fails the upgrade.
>
>The HV should be handling it gracefully else its a bug on the HV.
>If the HV handles it cleanly then CS can accordingly surface it and take
>corrective actions like updating the DB
>
>
>
>>
>>The idea is to not handle every possible scenario, but to ensure that the
>>vm (and system) is in a sane recoverable state after the unexpected
>>interaction.
>>
>>
>>On 1/21/13 10:01 PM, "Koushik Das" <ko...@citrix.com> wrote:
>>
>>>See inline for 1.
>>>
>>>-----Original Message-----
>>>From: Hari Kannan [mailto:hari.kannan@citrix.com]
>>>Sent: Tuesday, January 22, 2013 10:51 AM
>>>To: cloudstack-dev@incubator.apache.org
>>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>
>>>Hello Nitin, Koushik,
>>>
>>>I'm following up on this feature - is the FS located here still
>>>accurate/up to date?
>>>
>>>I also wish to get clarification on a couple of things:
>>>
>>>1.      There is a reference - open issue 1: "Ability to mark the VM for
>>>scale up at creation time " - what is the intent behind this capability?
>>>Why cant every VM be capable of scaling? Also, given the capability of
>>>scaling up is actually a property of {OS, Hypervisor} what would be the
>>>intent behind having this as a property of a service offering? How was
>>>this "closed"?
>>>
>>>[Koushik] For all HVs the ability to dynamically increase RAM/CPU needs
>>>to be explicitly enabled. This may mean that for some/all HVs there may
>>>be some overhead in terms of performance/capacity planning etc. (came
>>>across the following for Vmware
>>>http://www.yellow-bricks.com/2012/01/16/enabling-hot-add-by-default-cc-g
>>>a
>>>b
>>>virtualworld/). As a starting point I would like to have it enabled by
>>>default for all VMs. But later it may be required to attach some premium
>>>with this kind of offering.
>>>
>>>2.      We also know that XS and KVM support this for Linux (max needs
>>>to
>>>be pre-defined) - so, I assume we are supporting both these platforms,
>>>in
>>>addition to VMware?
>>>3.      In case there is no capacity in cluster to scale up, just making
>>>sure that the existing VM will not have any impact, right?
>>>
>>>Hari
>>>
>>>-----Original Message-----
>>>From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>Sent: Thursday, December 20, 2012 9:47 AM
>>>To: cloudstack-dev@incubator.apache.org
>>>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>
>>>Oh, if it's not already obvious, we're onboard for collaborating on this
>>>feature and can help implement the KVM hypervisor portions. :-)
>>>
>>>
>>>On Thu, Dec 20, 2012 at 8:44 AM, Marcus Sorensen
>>><sh...@gmail.com>wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Thu, Dec 20, 2012 at 4:52 AM, Koushik Das
>>>><ko...@citrix.com>wrote:
>>>>
>>>>> See inline
>>>>>
>>>>> Thanks,
>>>>> Koushik
>>>>>
>>>>> > -----Original Message-----
>>>>> > From: Chip Childers [mailto:chip.childers@sungard.com]
>>>>> > Sent: Wednesday, December 19, 2012 7:55 PM
>>>>> > To: cloudstack-dev@incubator.apache.org
>>>>> > Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>>> >
>>>>> > On Wed, Dec 19, 2012 at 3:34 AM, Koushik Das
>>>>> > <ko...@citrix.com>
>>>>> > wrote:
>>>>> > > See inline
>>>>> > >
>>>>> > >> -----Original Message-----
>>>>> > >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>> > >> Sent: Tuesday, December 18, 2012 10:35 PM
>>>>> > >> To: cloudstack-dev@incubator.apache.org
>>>>> > >> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>>> > >>
>>>>> > >> The FS looks good and addresses the things I'd want it to
>>>>> > >> (scaling should be limited to within cluster, use offerings).
>>>>> > >>
>>>>> > >> As you mention, there's a real problem surrounding no support
>>>>> > >> for scaling down CPU, and it's just as much a problem with the
>>>>> > >> guests as it is with hvms at the moment, it seems. This makes it
>>>>> > >> hard to just set a VM as a dynamic one, since at some point
>>>>> > >> you'll likely trigger it to scale up and have to reboot to get
>>>>> > >> back down. My suggestion if this goes through however is that
>>>>> > >> instead of marking a vm for auto scale, we can either attach
>>>>> > >> multiple compute offerings (with a priority or "level") to a VM,
>>>>> > >> along with triggers (we can't really
>>>>> trigger on
>>>>> > memory, but perhaps cpu utilization over a specific time, e.g.
>>>>> > >> if cpu is at 80% for x time, fall back to the next offering), or
>>>>> > >> we can create a specific single compute offering that allows you
>>>>> > >> to specify a min and max memory, cpu, and a trigger at which it
>>>>> > >> scales (this latter one is my preference).
>>>>> > >>
>>>>> > >> The whole thing is problematic though, because people can
>>>>> > >> inadvertently trigger their VM to scale up when they're
>>>>> > >> installing updates or compiling or something and then have to
>>>>> > >> reboot to come back down. If we can't take away resources
>>>>> > >> without manual intervention, we shouldn't add them. For this
>>>>> > >> reason I'd like to see the focus (at least initially) on simply
>>>>> > >> being able to change to larger compute offerings while the VM is
>>>>> > >> up. With this in place, if someone really wants to autoscale,
>>>>> > >> they can use the api in a combination of fetching the VM stats
>>>>> > >> and the existing changeServiceForVirtualMachine. Or we can put
>>>>> > >> that in, but I think
>>>>> any
>>>>> > implementation will be a poor experience without being able to go
>>>>> > both ways.
>>>>> > >>
>>>>> > >
>>>>> > > This is a good suggestion but as you have mentioned first
>>>>> > > priority is
>>>>> to have
>>>>> > the basic stuff working (increasing CPU/RAM for running VMs).
>>>>> > > Also another thing is that HVs (at least Vmware) require that a
>>>>> > > VM is
>>>>> > configured appropriately when it is stopped in order to support
>>>>> increasing
>>>>> > CPU/RAM while it is running. We can either do this for all VMs
>>>>> irrespective of
>>>>> > the fact whether the CPU/RAM is going to be actually increased OR
>>>>> > do it
>>>>> only
>>>>> > for selective VMs (maybe based on compute offering). If this is
>>>>> > going
>>>>> to be
>>>>> > common across all HVs the latter can be done.
>>>>>
>>>>
>>>> I think it could be done either way. The straightforward way is via
>>>> offering that allows for max/current CPU and max/current RAM to be
>>>> entered (basically exposing how the hypervisor settings themselves
>>>> work). But you could also do a global setting of some sort that says
>>>> 'set everything to a max of X CPU and Y RAM', so that every service
>>>> offering can be upgraded live. As you mention, it will require at
>>>> least a restart of the VMs to apply, so perhaps users could just
>>>> switch service offerings anyway. It could be handy to allow people to
>>>> upgrade service offering when it was unplanned for, though.
>>>>
>>>>
>>>>> > >
>>>>> > >> I don't know, maybe I'm off in left field here, I'd be
>>>>> > >> interested in hearing the thoughts of others.
>>>>> > >>
>>>>> > >> You mention  'upgradeVirtualMachine', which should be mentioned
>>>>> > >> on the customer facing API is called
>>>>> > >> 'changeServiceForVirtualMachine', just to reduce confusion.
>>>>> > >>
>>>>> > >
>>>>> > > upgradeVirtualMachine is an existing command (see
>>>>> > UpgradeVMCmd.java), was planning to reuse it. But yes if the name
>>>>> > sounds confusing we can deprecate it and create a new command with
>>>>> > the name you have suggested.
>>>>> > >
>>>>> >
>>>>> > Please don't break backward compatibility without the whole list
>>>>> discussing
>>>>> > the implications on a dedicated thread.  We had previously agreed
>>>>> > that
>>>>> we
>>>>> > were going to maintain API compatibility between 4.0.0-incubating
>>>>> > and
>>>>> our
>>>>> > next feature release.  If we break it, we have to release as
>>>>> 5.0.0-incubating
>>>>> > instead of 4.1.0-incubating.
>>>>>
>>>>> In that case will add a new async API changeServiceForVirtualMachine
>>>>> (or if anyone else comes up with a better name) which will work for
>>>>> both running and stopped VMs. upgradeVirtualMachine would continue to
>>>>> exist till
>>>>> 5.0.0 happens.
>>>>>
>>>>
>>>> Would this break backward compatibility? If an API call goes from
>>>> upgrading VMs only while they're off, and still upgrades VMs only
>>>> while they're off, but also upgrades VMs with a newer, specific
>>>> service offering type while they're on, does that break backward
>>>> compatibility? Or let's say we simply removed the check to make sure
>>>> the VM was off, and instead just checked if the VM was started with
>>>> the newer compatible settings... would that break backward
>>>> compatibility? The call still does what it did before when used as
>>>>before (changes service offering while the VM is off).
>>>>
>>>> Regarding upgradeVirtualMachine, I saw no mention of it in the API
>>>> docs, and found that in the code, changeServiceForVirtualMachine was
>>>> mapped to UpgradeVMCmd.java, which is why I mentioned the confusion.
>>>> 'upgradeVirtualMachine' only exists as an internal method of the
>>>> userVmService. See the file "client/tomcatconf/commands.properties.in"
>>>>
>>>> changeServiceForVirtualMachine=com.cloud.api.commands.UpgradeVMCmd
>>>>
>>>>
>>>>
>>>>>
>>>>> >
>>>>> > >>
>>>>> > >> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das
>>>>> > >> <koushik.das@citrix.com
>>>>> >
>>>>> > >> wrote:
>>>>> > >>
>>>>> > >> > Created first draft of the FS
>>>>> > >> >
>>>>> > >>
>>>>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scal
>>>>> > in
>>>>> > >> g
>>>>> > >> > +of+CPU+and+RAM
>>>>> > >> > Also created jira issue
>>>>> > >> > https://issues.apache.org/jira/browse/CLOUDSTACK-658
>>>>> > >> >
>>>>> > >> > Comments? There is an 'open issue' section where I have
>>>>> > >> > mentioned some issues that needs to be closed
>>>>> > >> >
>>>>> > >> > Thanks,
>>>>> > >> > Koushik
>>>>> > >> >
>>>>> > >> > > -----Original Message-----
>>>>> > >> > > From: Koushik Das [mailto:koushik.das@citrix.com]
>>>>> > >> > > Sent: Saturday, December 15, 2012 11:14 PM
>>>>> > >> > > To: cloudstack-dev@incubator.apache.org
>>>>> > >> > > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>>> > >> > >
>>>>> > >> > > Currently CS supports changing CPU and RAM for stopped VM.
>>>>> > >> > > This is achieved by changing compute offering of the VM
>>>>> > >> > > (with new CPU and RAM
>>>>> > >> > > values) and then starting it. I am planning to extend the
>>>>> > >> > > same for
>>>>> > >> > running VM
>>>>> > >> > > as well. Initially planning to do it for Vmware where CPU
>>>>> > >> > > and RAM can be dynamically increased. Support of other HVs
>>>>> > >> > > can also be added if they support increasing CPU/RAM.
>>>>> > >> > >
>>>>> > >> > > Assuming that in the updated compute offering only CPU and
>>>>> > >> > > RAM has changed, the deployment planner can either select
>>>>> > >> > > the same host in which case the values are dynamically
>>>>> > >> > > scaled up OR a different one in which
>>>>> > >> > case
>>>>> > >> > > the operation fails. In future if there is support for live
>>>>> > >> > > migration
>>>>> > >> > (provided
>>>>> > >> > > HV supports it) then another option in the latter case could
>>>>> > >> > > be to
>>>>> > >> > migrate the
>>>>> > >> > > VM first and then scale it up.
>>>>> > >> > >
>>>>> > >> > > I will start working on the FS and share it out sometime
>>>>> > >> > > next
>>>>> week.
>>>>> > >> > >
>>>>> > >> > > Comments/suggestions?
>>>>> > >> > >
>>>>> > >> > > Thanks,
>>>>> > >> > > Koushik
>>>>> > >> >
>>>>> > >
>>>>>
>>>>
>>>>
>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Nitin Mehta <Ni...@citrix.com>]

Chiradeep - thanks for your questions. Please find my answers inline. I
might not have the best solutions for some of them so looking for some
guidance as well.
All these interactions make for good test cases so the QA for this feature
should test these interactions.


On 22/01/13 12:18 PM, "Chiradeep Vittal" <Ch...@citrix.com>
wrote:

>What usage events will be generated to ensure that the proper billing is
>done?

Currently usage implementation won't be able to handle this. I will
introduce a 
new event to handle it for dynamically as well as statically(for Hvs that
do not support this)

>We need more details about the actual classes being affected and the
>layers at which the orchestration is being done.

I will put that information in the FS. It will be based on the flowchart
in the FS.

>What are the possible interactions that you are taking care of?
> - user powers off the vm during the operation

Need to explore how the HV is handling it. The HV should be handling it
gracefully else its a bug on the HV.
If the HV handles it cleanly then CS can accordingly surface it and take
corrective actions like updating the DB etc.

> - race conditions during calculation of sufficient capacity

It will be implemented parallel to how we do it in allocators. As soon as
we find a suitable destination we lock the capacity and release it in case
of 
failures. All this is through state machine transitions so will be done
cleanly.

> - failure during live migration

As you see in the flowchart in the FS I won't be touching the live
migration but leveraging on the current implementation.
I am assuming that we already handle it gracefully. But will definitely
test it.

> - HA event during live migration / upgrade

I am assuming that we already handle the HA event during live migrations
since this scenario is available in current implementation.
Will try and leverage on the same for HA event during vm upgrade.

> - scheduled snapshot of volumes during the operation

For vmware, the entire vm is locked by HV and this can be an issue. I will
leverage on current implementations for existing interactions like
scheduled snapshots events during live migration and will replicate the
same.
  

> - attach / detach during the operation

Same thing as the above.

> - hypervisor fails the upgrade.

The HV should be handling it gracefully else its a bug on the HV.
If the HV handles it cleanly then CS can accordingly surface it and take
corrective actions like updating the DB



>
>The idea is to not handle every possible scenario, but to ensure that the
>vm (and system) is in a sane recoverable state after the unexpected
>interaction.
> 
>
>On 1/21/13 10:01 PM, "Koushik Das" <ko...@citrix.com> wrote:
>
>>See inline for 1.
>>
>>-----Original Message-----
>>From: Hari Kannan [mailto:hari.kannan@citrix.com]
>>Sent: Tuesday, January 22, 2013 10:51 AM
>>To: cloudstack-dev@incubator.apache.org
>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>>
>>Hello Nitin, Koushik,
>>
>>I'm following up on this feature - is the FS located here still
>>accurate/up to date?
>>
>>I also wish to get clarification on a couple of things:
>>
>>1.      There is a reference - open issue 1: "Ability to mark the VM for
>>scale up at creation time " - what is the intent behind this capability?
>>Why cant every VM be capable of scaling? Also, given the capability of
>>scaling up is actually a property of {OS, Hypervisor} what would be the
>>intent behind having this as a property of a service offering? How was
>>this "closed"?
>>
>>[Koushik] For all HVs the ability to dynamically increase RAM/CPU needs
>>to be explicitly enabled. This may mean that for some/all HVs there may
>>be some overhead in terms of performance/capacity planning etc. (came
>>across the following for Vmware
>>http://www.yellow-bricks.com/2012/01/16/enabling-hot-add-by-default-cc-ga
>>b
>>virtualworld/). As a starting point I would like to have it enabled by
>>default for all VMs. But later it may be required to attach some premium
>>with this kind of offering.
>>
>>2.      We also know that XS and KVM support this for Linux (max needs to
>>be pre-defined) - so, I assume we are supporting both these platforms, in
>>addition to VMware?
>>3.      In case there is no capacity in cluster to scale up, just making
>>sure that the existing VM will not have any impact, right?
>>
>>Hari
>>
>>-----Original Message-----
>>From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>Sent: Thursday, December 20, 2012 9:47 AM
>>To: cloudstack-dev@incubator.apache.org
>>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>
>>Oh, if it's not already obvious, we're onboard for collaborating on this
>>feature and can help implement the KVM hypervisor portions. :-)
>>
>>
>>On Thu, Dec 20, 2012 at 8:44 AM, Marcus Sorensen
>><sh...@gmail.com>wrote:
>>
>>>
>>>
>>>
>>> On Thu, Dec 20, 2012 at 4:52 AM, Koushik Das
>>><ko...@citrix.com>wrote:
>>>
>>>> See inline
>>>>
>>>> Thanks,
>>>> Koushik
>>>>
>>>> > -----Original Message-----
>>>> > From: Chip Childers [mailto:chip.childers@sungard.com]
>>>> > Sent: Wednesday, December 19, 2012 7:55 PM
>>>> > To: cloudstack-dev@incubator.apache.org
>>>> > Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>> >
>>>> > On Wed, Dec 19, 2012 at 3:34 AM, Koushik Das
>>>> > <ko...@citrix.com>
>>>> > wrote:
>>>> > > See inline
>>>> > >
>>>> > >> -----Original Message-----
>>>> > >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>> > >> Sent: Tuesday, December 18, 2012 10:35 PM
>>>> > >> To: cloudstack-dev@incubator.apache.org
>>>> > >> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>> > >>
>>>> > >> The FS looks good and addresses the things I'd want it to
>>>> > >> (scaling should be limited to within cluster, use offerings).
>>>> > >>
>>>> > >> As you mention, there's a real problem surrounding no support
>>>> > >> for scaling down CPU, and it's just as much a problem with the
>>>> > >> guests as it is with hvms at the moment, it seems. This makes it
>>>> > >> hard to just set a VM as a dynamic one, since at some point
>>>> > >> you'll likely trigger it to scale up and have to reboot to get
>>>> > >> back down. My suggestion if this goes through however is that
>>>> > >> instead of marking a vm for auto scale, we can either attach
>>>> > >> multiple compute offerings (with a priority or "level") to a VM,
>>>> > >> along with triggers (we can't really
>>>> trigger on
>>>> > memory, but perhaps cpu utilization over a specific time, e.g.
>>>> > >> if cpu is at 80% for x time, fall back to the next offering), or
>>>> > >> we can create a specific single compute offering that allows you
>>>> > >> to specify a min and max memory, cpu, and a trigger at which it
>>>> > >> scales (this latter one is my preference).
>>>> > >>
>>>> > >> The whole thing is problematic though, because people can
>>>> > >> inadvertently trigger their VM to scale up when they're
>>>> > >> installing updates or compiling or something and then have to
>>>> > >> reboot to come back down. If we can't take away resources
>>>> > >> without manual intervention, we shouldn't add them. For this
>>>> > >> reason I'd like to see the focus (at least initially) on simply
>>>> > >> being able to change to larger compute offerings while the VM is
>>>> > >> up. With this in place, if someone really wants to autoscale,
>>>> > >> they can use the api in a combination of fetching the VM stats
>>>> > >> and the existing changeServiceForVirtualMachine. Or we can put
>>>> > >> that in, but I think
>>>> any
>>>> > implementation will be a poor experience without being able to go
>>>> > both ways.
>>>> > >>
>>>> > >
>>>> > > This is a good suggestion but as you have mentioned first
>>>> > > priority is
>>>> to have
>>>> > the basic stuff working (increasing CPU/RAM for running VMs).
>>>> > > Also another thing is that HVs (at least Vmware) require that a
>>>> > > VM is
>>>> > configured appropriately when it is stopped in order to support
>>>> increasing
>>>> > CPU/RAM while it is running. We can either do this for all VMs
>>>> irrespective of
>>>> > the fact whether the CPU/RAM is going to be actually increased OR
>>>> > do it
>>>> only
>>>> > for selective VMs (maybe based on compute offering). If this is
>>>> > going
>>>> to be
>>>> > common across all HVs the latter can be done.
>>>>
>>>
>>> I think it could be done either way. The straightforward way is via
>>> offering that allows for max/current CPU and max/current RAM to be
>>> entered (basically exposing how the hypervisor settings themselves
>>> work). But you could also do a global setting of some sort that says
>>> 'set everything to a max of X CPU and Y RAM', so that every service
>>> offering can be upgraded live. As you mention, it will require at
>>> least a restart of the VMs to apply, so perhaps users could just
>>> switch service offerings anyway. It could be handy to allow people to
>>> upgrade service offering when it was unplanned for, though.
>>>
>>>
>>>> > >
>>>> > >> I don't know, maybe I'm off in left field here, I'd be
>>>> > >> interested in hearing the thoughts of others.
>>>> > >>
>>>> > >> You mention  'upgradeVirtualMachine', which should be mentioned
>>>> > >> on the customer facing API is called
>>>> > >> 'changeServiceForVirtualMachine', just to reduce confusion.
>>>> > >>
>>>> > >
>>>> > > upgradeVirtualMachine is an existing command (see
>>>> > UpgradeVMCmd.java), was planning to reuse it. But yes if the name
>>>> > sounds confusing we can deprecate it and create a new command with
>>>> > the name you have suggested.
>>>> > >
>>>> >
>>>> > Please don't break backward compatibility without the whole list
>>>> discussing
>>>> > the implications on a dedicated thread.  We had previously agreed
>>>> > that
>>>> we
>>>> > were going to maintain API compatibility between 4.0.0-incubating
>>>> > and
>>>> our
>>>> > next feature release.  If we break it, we have to release as
>>>> 5.0.0-incubating
>>>> > instead of 4.1.0-incubating.
>>>>
>>>> In that case will add a new async API changeServiceForVirtualMachine
>>>> (or if anyone else comes up with a better name) which will work for
>>>> both running and stopped VMs. upgradeVirtualMachine would continue to
>>>> exist till
>>>> 5.0.0 happens.
>>>>
>>>
>>> Would this break backward compatibility? If an API call goes from
>>> upgrading VMs only while they're off, and still upgrades VMs only
>>> while they're off, but also upgrades VMs with a newer, specific
>>> service offering type while they're on, does that break backward
>>> compatibility? Or let's say we simply removed the check to make sure
>>> the VM was off, and instead just checked if the VM was started with
>>> the newer compatible settings... would that break backward
>>> compatibility? The call still does what it did before when used as
>>>before (changes service offering while the VM is off).
>>>
>>> Regarding upgradeVirtualMachine, I saw no mention of it in the API
>>> docs, and found that in the code, changeServiceForVirtualMachine was
>>> mapped to UpgradeVMCmd.java, which is why I mentioned the confusion.
>>> 'upgradeVirtualMachine' only exists as an internal method of the
>>> userVmService. See the file "client/tomcatconf/commands.properties.in"
>>>
>>> changeServiceForVirtualMachine=com.cloud.api.commands.UpgradeVMCmd
>>>
>>>
>>>
>>>>
>>>> >
>>>> > >>
>>>> > >> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das
>>>> > >> <koushik.das@citrix.com
>>>> >
>>>> > >> wrote:
>>>> > >>
>>>> > >> > Created first draft of the FS
>>>> > >> >
>>>> > >>
>>>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scal
>>>> > in
>>>> > >> g
>>>> > >> > +of+CPU+and+RAM
>>>> > >> > Also created jira issue
>>>> > >> > https://issues.apache.org/jira/browse/CLOUDSTACK-658
>>>> > >> >
>>>> > >> > Comments? There is an 'open issue' section where I have
>>>> > >> > mentioned some issues that needs to be closed
>>>> > >> >
>>>> > >> > Thanks,
>>>> > >> > Koushik
>>>> > >> >
>>>> > >> > > -----Original Message-----
>>>> > >> > > From: Koushik Das [mailto:koushik.das@citrix.com]
>>>> > >> > > Sent: Saturday, December 15, 2012 11:14 PM
>>>> > >> > > To: cloudstack-dev@incubator.apache.org
>>>> > >> > > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>> > >> > >
>>>> > >> > > Currently CS supports changing CPU and RAM for stopped VM.
>>>> > >> > > This is achieved by changing compute offering of the VM
>>>> > >> > > (with new CPU and RAM
>>>> > >> > > values) and then starting it. I am planning to extend the
>>>> > >> > > same for
>>>> > >> > running VM
>>>> > >> > > as well. Initially planning to do it for Vmware where CPU
>>>> > >> > > and RAM can be dynamically increased. Support of other HVs
>>>> > >> > > can also be added if they support increasing CPU/RAM.
>>>> > >> > >
>>>> > >> > > Assuming that in the updated compute offering only CPU and
>>>> > >> > > RAM has changed, the deployment planner can either select
>>>> > >> > > the same host in which case the values are dynamically
>>>> > >> > > scaled up OR a different one in which
>>>> > >> > case
>>>> > >> > > the operation fails. In future if there is support for live
>>>> > >> > > migration
>>>> > >> > (provided
>>>> > >> > > HV supports it) then another option in the latter case could
>>>> > >> > > be to
>>>> > >> > migrate the
>>>> > >> > > VM first and then scale it up.
>>>> > >> > >
>>>> > >> > > I will start working on the FS and share it out sometime
>>>> > >> > > next
>>>> week.
>>>> > >> > >
>>>> > >> > > Comments/suggestions?
>>>> > >> > >
>>>> > >> > > Thanks,
>>>> > >> > > Koushik
>>>> > >> >
>>>> > >
>>>>
>>>
>>>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Chiradeep Vittal <Ch...@citrix.com>]

What usage events will be generated to ensure that the proper billing is
done?
We need more details about the actual classes being affected and the
layers at which the orchestration is being done.
What are the possible interactions that you are taking care of?
 - user powers off the vm during the operation
 - race conditions during calculation of sufficient capacity
 - failure during live migration
 - HA event during live migration / upgrade
 - scheduled snapshot of volumes during the operation
 - attach / detach during the operation
 - hypervisor fails the upgrade.

The idea is to not handle every possible scenario, but to ensure that the
vm (and system) is in a sane recoverable state after the unexpected
interaction.
 

On 1/21/13 10:01 PM, "Koushik Das" <ko...@citrix.com> wrote:

>See inline for 1.
>
>-----Original Message-----
>From: Hari Kannan [mailto:hari.kannan@citrix.com]
>Sent: Tuesday, January 22, 2013 10:51 AM
>To: cloudstack-dev@incubator.apache.org
>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>
>Hello Nitin, Koushik,
>
>I'm following up on this feature - is the FS located here still
>accurate/up to date?
>
>I also wish to get clarification on a couple of things:
>
>1.      There is a reference - open issue 1: "Ability to mark the VM for
>scale up at creation time " - what is the intent behind this capability?
>Why cant every VM be capable of scaling? Also, given the capability of
>scaling up is actually a property of {OS, Hypervisor} what would be the
>intent behind having this as a property of a service offering? How was
>this "closed"?
>
>[Koushik] For all HVs the ability to dynamically increase RAM/CPU needs
>to be explicitly enabled. This may mean that for some/all HVs there may
>be some overhead in terms of performance/capacity planning etc. (came
>across the following for Vmware
>http://www.yellow-bricks.com/2012/01/16/enabling-hot-add-by-default-cc-gab
>virtualworld/). As a starting point I would like to have it enabled by
>default for all VMs. But later it may be required to attach some premium
>with this kind of offering.
>
>2.      We also know that XS and KVM support this for Linux (max needs to
>be pre-defined) - so, I assume we are supporting both these platforms, in
>addition to VMware?
>3.      In case there is no capacity in cluster to scale up, just making
>sure that the existing VM will not have any impact, right?
>
>Hari
>
>-----Original Message-----
>From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>Sent: Thursday, December 20, 2012 9:47 AM
>To: cloudstack-dev@incubator.apache.org
>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>
>Oh, if it's not already obvious, we're onboard for collaborating on this
>feature and can help implement the KVM hypervisor portions. :-)
>
>
>On Thu, Dec 20, 2012 at 8:44 AM, Marcus Sorensen
><sh...@gmail.com>wrote:
>
>>
>>
>>
>> On Thu, Dec 20, 2012 at 4:52 AM, Koushik Das
>><ko...@citrix.com>wrote:
>>
>>> See inline
>>>
>>> Thanks,
>>> Koushik
>>>
>>> > -----Original Message-----
>>> > From: Chip Childers [mailto:chip.childers@sungard.com]
>>> > Sent: Wednesday, December 19, 2012 7:55 PM
>>> > To: cloudstack-dev@incubator.apache.org
>>> > Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> >
>>> > On Wed, Dec 19, 2012 at 3:34 AM, Koushik Das
>>> > <ko...@citrix.com>
>>> > wrote:
>>> > > See inline
>>> > >
>>> > >> -----Original Message-----
>>> > >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>> > >> Sent: Tuesday, December 18, 2012 10:35 PM
>>> > >> To: cloudstack-dev@incubator.apache.org
>>> > >> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> > >>
>>> > >> The FS looks good and addresses the things I'd want it to
>>> > >> (scaling should be limited to within cluster, use offerings).
>>> > >>
>>> > >> As you mention, there's a real problem surrounding no support
>>> > >> for scaling down CPU, and it's just as much a problem with the
>>> > >> guests as it is with hvms at the moment, it seems. This makes it
>>> > >> hard to just set a VM as a dynamic one, since at some point
>>> > >> you'll likely trigger it to scale up and have to reboot to get
>>> > >> back down. My suggestion if this goes through however is that
>>> > >> instead of marking a vm for auto scale, we can either attach
>>> > >> multiple compute offerings (with a priority or "level") to a VM,
>>> > >> along with triggers (we can't really
>>> trigger on
>>> > memory, but perhaps cpu utilization over a specific time, e.g.
>>> > >> if cpu is at 80% for x time, fall back to the next offering), or
>>> > >> we can create a specific single compute offering that allows you
>>> > >> to specify a min and max memory, cpu, and a trigger at which it
>>> > >> scales (this latter one is my preference).
>>> > >>
>>> > >> The whole thing is problematic though, because people can
>>> > >> inadvertently trigger their VM to scale up when they're
>>> > >> installing updates or compiling or something and then have to
>>> > >> reboot to come back down. If we can't take away resources
>>> > >> without manual intervention, we shouldn't add them. For this
>>> > >> reason I'd like to see the focus (at least initially) on simply
>>> > >> being able to change to larger compute offerings while the VM is
>>> > >> up. With this in place, if someone really wants to autoscale,
>>> > >> they can use the api in a combination of fetching the VM stats
>>> > >> and the existing changeServiceForVirtualMachine. Or we can put
>>> > >> that in, but I think
>>> any
>>> > implementation will be a poor experience without being able to go
>>> > both ways.
>>> > >>
>>> > >
>>> > > This is a good suggestion but as you have mentioned first
>>> > > priority is
>>> to have
>>> > the basic stuff working (increasing CPU/RAM for running VMs).
>>> > > Also another thing is that HVs (at least Vmware) require that a
>>> > > VM is
>>> > configured appropriately when it is stopped in order to support
>>> increasing
>>> > CPU/RAM while it is running. We can either do this for all VMs
>>> irrespective of
>>> > the fact whether the CPU/RAM is going to be actually increased OR
>>> > do it
>>> only
>>> > for selective VMs (maybe based on compute offering). If this is
>>> > going
>>> to be
>>> > common across all HVs the latter can be done.
>>>
>>
>> I think it could be done either way. The straightforward way is via
>> offering that allows for max/current CPU and max/current RAM to be
>> entered (basically exposing how the hypervisor settings themselves
>> work). But you could also do a global setting of some sort that says
>> 'set everything to a max of X CPU and Y RAM', so that every service
>> offering can be upgraded live. As you mention, it will require at
>> least a restart of the VMs to apply, so perhaps users could just
>> switch service offerings anyway. It could be handy to allow people to
>> upgrade service offering when it was unplanned for, though.
>>
>>
>>> > >
>>> > >> I don't know, maybe I'm off in left field here, I'd be
>>> > >> interested in hearing the thoughts of others.
>>> > >>
>>> > >> You mention  'upgradeVirtualMachine', which should be mentioned
>>> > >> on the customer facing API is called
>>> > >> 'changeServiceForVirtualMachine', just to reduce confusion.
>>> > >>
>>> > >
>>> > > upgradeVirtualMachine is an existing command (see
>>> > UpgradeVMCmd.java), was planning to reuse it. But yes if the name
>>> > sounds confusing we can deprecate it and create a new command with
>>> > the name you have suggested.
>>> > >
>>> >
>>> > Please don't break backward compatibility without the whole list
>>> discussing
>>> > the implications on a dedicated thread.  We had previously agreed
>>> > that
>>> we
>>> > were going to maintain API compatibility between 4.0.0-incubating
>>> > and
>>> our
>>> > next feature release.  If we break it, we have to release as
>>> 5.0.0-incubating
>>> > instead of 4.1.0-incubating.
>>>
>>> In that case will add a new async API changeServiceForVirtualMachine
>>> (or if anyone else comes up with a better name) which will work for
>>> both running and stopped VMs. upgradeVirtualMachine would continue to
>>> exist till
>>> 5.0.0 happens.
>>>
>>
>> Would this break backward compatibility? If an API call goes from
>> upgrading VMs only while they're off, and still upgrades VMs only
>> while they're off, but also upgrades VMs with a newer, specific
>> service offering type while they're on, does that break backward
>> compatibility? Or let's say we simply removed the check to make sure
>> the VM was off, and instead just checked if the VM was started with
>> the newer compatible settings... would that break backward
>> compatibility? The call still does what it did before when used as
>>before (changes service offering while the VM is off).
>>
>> Regarding upgradeVirtualMachine, I saw no mention of it in the API
>> docs, and found that in the code, changeServiceForVirtualMachine was
>> mapped to UpgradeVMCmd.java, which is why I mentioned the confusion.
>> 'upgradeVirtualMachine' only exists as an internal method of the
>> userVmService. See the file "client/tomcatconf/commands.properties.in"
>>
>> changeServiceForVirtualMachine=com.cloud.api.commands.UpgradeVMCmd
>>
>>
>>
>>>
>>> >
>>> > >>
>>> > >> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das
>>> > >> <koushik.das@citrix.com
>>> >
>>> > >> wrote:
>>> > >>
>>> > >> > Created first draft of the FS
>>> > >> >
>>> > >>
>>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scal
>>> > in
>>> > >> g
>>> > >> > +of+CPU+and+RAM
>>> > >> > Also created jira issue
>>> > >> > https://issues.apache.org/jira/browse/CLOUDSTACK-658
>>> > >> >
>>> > >> > Comments? There is an 'open issue' section where I have
>>> > >> > mentioned some issues that needs to be closed
>>> > >> >
>>> > >> > Thanks,
>>> > >> > Koushik
>>> > >> >
>>> > >> > > -----Original Message-----
>>> > >> > > From: Koushik Das [mailto:koushik.das@citrix.com]
>>> > >> > > Sent: Saturday, December 15, 2012 11:14 PM
>>> > >> > > To: cloudstack-dev@incubator.apache.org
>>> > >> > > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> > >> > >
>>> > >> > > Currently CS supports changing CPU and RAM for stopped VM.
>>> > >> > > This is achieved by changing compute offering of the VM
>>> > >> > > (with new CPU and RAM
>>> > >> > > values) and then starting it. I am planning to extend the
>>> > >> > > same for
>>> > >> > running VM
>>> > >> > > as well. Initially planning to do it for Vmware where CPU
>>> > >> > > and RAM can be dynamically increased. Support of other HVs
>>> > >> > > can also be added if they support increasing CPU/RAM.
>>> > >> > >
>>> > >> > > Assuming that in the updated compute offering only CPU and
>>> > >> > > RAM has changed, the deployment planner can either select
>>> > >> > > the same host in which case the values are dynamically
>>> > >> > > scaled up OR a different one in which
>>> > >> > case
>>> > >> > > the operation fails. In future if there is support for live
>>> > >> > > migration
>>> > >> > (provided
>>> > >> > > HV supports it) then another option in the latter case could
>>> > >> > > be to
>>> > >> > migrate the
>>> > >> > > VM first and then scale it up.
>>> > >> > >
>>> > >> > > I will start working on the FS and share it out sometime
>>> > >> > > next
>>> week.
>>> > >> > >
>>> > >> > > Comments/suggestions?
>>> > >> > >
>>> > >> > > Thanks,
>>> > >> > > Koushik
>>> > >> >
>>> > >
>>>
>>
>>

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Koushik Das <ko...@citrix.com>]

See inline for 1.

-----Original Message-----
From: Hari Kannan [mailto:hari.kannan@citrix.com] 
Sent: Tuesday, January 22, 2013 10:51 AM
To: cloudstack-dev@incubator.apache.org
Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs

Hello Nitin, Koushik,

I'm following up on this feature - is the FS located here still accurate/up to date?

I also wish to get clarification on a couple of things:

1.	There is a reference - open issue 1: "Ability to mark the VM for scale up at creation time " - what is the intent behind this capability? Why cant every VM be capable of scaling? Also, given the capability of scaling up is actually a property of {OS, Hypervisor} what would be the intent behind having this as a property of a service offering? How was this "closed"?

[Koushik] For all HVs the ability to dynamically increase RAM/CPU needs to be explicitly enabled. This may mean that for some/all HVs there may be some overhead in terms of performance/capacity planning etc. (came across the following for Vmware http://www.yellow-bricks.com/2012/01/16/enabling-hot-add-by-default-cc-gabvirtualworld/). As a starting point I would like to have it enabled by default for all VMs. But later it may be required to attach some premium with this kind of offering.

2.	We also know that XS and KVM support this for Linux (max needs to be pre-defined) - so, I assume we are supporting both these platforms, in addition to VMware?
3.	In case there is no capacity in cluster to scale up, just making sure that the existing VM will not have any impact, right?

Hari

-----Original Message-----
From: Marcus Sorensen [mailto:shadowsor@gmail.com]
Sent: Thursday, December 20, 2012 9:47 AM
To: cloudstack-dev@incubator.apache.org
Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs

Oh, if it's not already obvious, we're onboard for collaborating on this feature and can help implement the KVM hypervisor portions. :-)


On Thu, Dec 20, 2012 at 8:44 AM, Marcus Sorensen <sh...@gmail.com>wrote:

>
>
>
> On Thu, Dec 20, 2012 at 4:52 AM, Koushik Das <ko...@citrix.com>wrote:
>
>> See inline
>>
>> Thanks,
>> Koushik
>>
>> > -----Original Message-----
>> > From: Chip Childers [mailto:chip.childers@sungard.com]
>> > Sent: Wednesday, December 19, 2012 7:55 PM
>> > To: cloudstack-dev@incubator.apache.org
>> > Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>> >
>> > On Wed, Dec 19, 2012 at 3:34 AM, Koushik Das 
>> > <ko...@citrix.com>
>> > wrote:
>> > > See inline
>> > >
>> > >> -----Original Message-----
>> > >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> > >> Sent: Tuesday, December 18, 2012 10:35 PM
>> > >> To: cloudstack-dev@incubator.apache.org
>> > >> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>> > >>
>> > >> The FS looks good and addresses the things I'd want it to 
>> > >> (scaling should be limited to within cluster, use offerings).
>> > >>
>> > >> As you mention, there's a real problem surrounding no support 
>> > >> for scaling down CPU, and it's just as much a problem with the 
>> > >> guests as it is with hvms at the moment, it seems. This makes it 
>> > >> hard to just set a VM as a dynamic one, since at some point 
>> > >> you'll likely trigger it to scale up and have to reboot to get 
>> > >> back down. My suggestion if this goes through however is that 
>> > >> instead of marking a vm for auto scale, we can either attach 
>> > >> multiple compute offerings (with a priority or "level") to a VM, 
>> > >> along with triggers (we can't really
>> trigger on
>> > memory, but perhaps cpu utilization over a specific time, e.g.
>> > >> if cpu is at 80% for x time, fall back to the next offering), or 
>> > >> we can create a specific single compute offering that allows you 
>> > >> to specify a min and max memory, cpu, and a trigger at which it 
>> > >> scales (this latter one is my preference).
>> > >>
>> > >> The whole thing is problematic though, because people can 
>> > >> inadvertently trigger their VM to scale up when they're 
>> > >> installing updates or compiling or something and then have to 
>> > >> reboot to come back down. If we can't take away resources 
>> > >> without manual intervention, we shouldn't add them. For this 
>> > >> reason I'd like to see the focus (at least initially) on simply 
>> > >> being able to change to larger compute offerings while the VM is 
>> > >> up. With this in place, if someone really wants to autoscale, 
>> > >> they can use the api in a combination of fetching the VM stats 
>> > >> and the existing changeServiceForVirtualMachine. Or we can put 
>> > >> that in, but I think
>> any
>> > implementation will be a poor experience without being able to go 
>> > both ways.
>> > >>
>> > >
>> > > This is a good suggestion but as you have mentioned first 
>> > > priority is
>> to have
>> > the basic stuff working (increasing CPU/RAM for running VMs).
>> > > Also another thing is that HVs (at least Vmware) require that a 
>> > > VM is
>> > configured appropriately when it is stopped in order to support
>> increasing
>> > CPU/RAM while it is running. We can either do this for all VMs
>> irrespective of
>> > the fact whether the CPU/RAM is going to be actually increased OR 
>> > do it
>> only
>> > for selective VMs (maybe based on compute offering). If this is 
>> > going
>> to be
>> > common across all HVs the latter can be done.
>>
>
> I think it could be done either way. The straightforward way is via 
> offering that allows for max/current CPU and max/current RAM to be 
> entered (basically exposing how the hypervisor settings themselves 
> work). But you could also do a global setting of some sort that says 
> 'set everything to a max of X CPU and Y RAM', so that every service 
> offering can be upgraded live. As you mention, it will require at 
> least a restart of the VMs to apply, so perhaps users could just 
> switch service offerings anyway. It could be handy to allow people to 
> upgrade service offering when it was unplanned for, though.
>
>
>> > >
>> > >> I don't know, maybe I'm off in left field here, I'd be 
>> > >> interested in hearing the thoughts of others.
>> > >>
>> > >> You mention  'upgradeVirtualMachine', which should be mentioned 
>> > >> on the customer facing API is called 
>> > >> 'changeServiceForVirtualMachine', just to reduce confusion.
>> > >>
>> > >
>> > > upgradeVirtualMachine is an existing command (see
>> > UpgradeVMCmd.java), was planning to reuse it. But yes if the name 
>> > sounds confusing we can deprecate it and create a new command with 
>> > the name you have suggested.
>> > >
>> >
>> > Please don't break backward compatibility without the whole list
>> discussing
>> > the implications on a dedicated thread.  We had previously agreed 
>> > that
>> we
>> > were going to maintain API compatibility between 4.0.0-incubating 
>> > and
>> our
>> > next feature release.  If we break it, we have to release as
>> 5.0.0-incubating
>> > instead of 4.1.0-incubating.
>>
>> In that case will add a new async API changeServiceForVirtualMachine 
>> (or if anyone else comes up with a better name) which will work for 
>> both running and stopped VMs. upgradeVirtualMachine would continue to 
>> exist till
>> 5.0.0 happens.
>>
>
> Would this break backward compatibility? If an API call goes from 
> upgrading VMs only while they're off, and still upgrades VMs only 
> while they're off, but also upgrades VMs with a newer, specific 
> service offering type while they're on, does that break backward 
> compatibility? Or let's say we simply removed the check to make sure 
> the VM was off, and instead just checked if the VM was started with 
> the newer compatible settings... would that break backward 
> compatibility? The call still does what it did before when used as before (changes service offering while the VM is off).
>
> Regarding upgradeVirtualMachine, I saw no mention of it in the API 
> docs, and found that in the code, changeServiceForVirtualMachine was 
> mapped to UpgradeVMCmd.java, which is why I mentioned the confusion.
> 'upgradeVirtualMachine' only exists as an internal method of the 
> userVmService. See the file "client/tomcatconf/commands.properties.in"
>
> changeServiceForVirtualMachine=com.cloud.api.commands.UpgradeVMCmd
>
>
>
>>
>> >
>> > >>
>> > >> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das 
>> > >> <koushik.das@citrix.com
>> >
>> > >> wrote:
>> > >>
>> > >> > Created first draft of the FS
>> > >> >
>> > >>
>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scal
>> > in
>> > >> g
>> > >> > +of+CPU+and+RAM
>> > >> > Also created jira issue
>> > >> > https://issues.apache.org/jira/browse/CLOUDSTACK-658
>> > >> >
>> > >> > Comments? There is an 'open issue' section where I have 
>> > >> > mentioned some issues that needs to be closed
>> > >> >
>> > >> > Thanks,
>> > >> > Koushik
>> > >> >
>> > >> > > -----Original Message-----
>> > >> > > From: Koushik Das [mailto:koushik.das@citrix.com]
>> > >> > > Sent: Saturday, December 15, 2012 11:14 PM
>> > >> > > To: cloudstack-dev@incubator.apache.org
>> > >> > > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>> > >> > >
>> > >> > > Currently CS supports changing CPU and RAM for stopped VM. 
>> > >> > > This is achieved by changing compute offering of the VM 
>> > >> > > (with new CPU and RAM
>> > >> > > values) and then starting it. I am planning to extend the 
>> > >> > > same for
>> > >> > running VM
>> > >> > > as well. Initially planning to do it for Vmware where CPU 
>> > >> > > and RAM can be dynamically increased. Support of other HVs 
>> > >> > > can also be added if they support increasing CPU/RAM.
>> > >> > >
>> > >> > > Assuming that in the updated compute offering only CPU and 
>> > >> > > RAM has changed, the deployment planner can either select 
>> > >> > > the same host in which case the values are dynamically 
>> > >> > > scaled up OR a different one in which
>> > >> > case
>> > >> > > the operation fails. In future if there is support for live 
>> > >> > > migration
>> > >> > (provided
>> > >> > > HV supports it) then another option in the latter case could 
>> > >> > > be to
>> > >> > migrate the
>> > >> > > VM first and then scale it up.
>> > >> > >
>> > >> > > I will start working on the FS and share it out sometime 
>> > >> > > next
>> week.
>> > >> > >
>> > >> > > Comments/suggestions?
>> > >> > >
>> > >> > > Thanks,
>> > >> > > Koushik
>> > >> >
>> > >
>>
>
>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Wido den Hollander <wi...@widodh.nl>]

On 01/22/2013 07:00 AM, Nitin Mehta wrote:
> Hari - I am yet to update the FS. I will do that very soon. Answer to your
> questions inline.
>
> On 22/01/13 10:50 AM, "Hari Kannan" <ha...@citrix.com> wrote:
>
>> Hello Nitin, Koushik,
>>
>> I'm following up on this feature - is the FS located here still
>> accurate/up to date?
>>
>> I also wish to get clarification on a couple of things:
>>
>> 1.	There is a reference - open issue 1: "Ability to mark the VM for scale
>> up at creation time " - what is the intent behind this capability? Why
>> cant every VM be capable of scaling? Also, given the capability of
>> scaling up is actually a property of {OS, Hypervisor} what would be the
>> intent behind having this as a property of a service offering? How was
>> this "closed"?
>
> We would not be doing this. The intent earlier was because Vmware required
> this property to be set upfront (while deploying vm itself). We will set
> this property for every vm be default.
>
>> 2.	We also know that XS and KVM support this for Linux (max needs to be
>> pre-defined) - so, I assume we are supporting both these platforms, in
>> addition to VMware?
>
> I am not planning to do this. I think for KVM Marcus was going to pitch in.
>

For KVM you indeed need to set "maxmem" and "currentmem". Maxmem is the 
value until which you can increase the memory while the VM is running. 
If you want to increase beyond that you need to restart the VM.

Wido

>> 3.	In case there is no capacity in cluster to scale up, just making sure
>> that the existing VM will not have any impact, right?
>
> Yes, there won't be any impact. Don't worry about it :)
>
>>
>> Hari
>>
>> -----Original Message-----
>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> Sent: Thursday, December 20, 2012 9:47 AM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>
>> Oh, if it's not already obvious, we're onboard for collaborating on this
>> feature and can help implement the KVM hypervisor portions. :-)
>>
>>
>> On Thu, Dec 20, 2012 at 8:44 AM, Marcus Sorensen
>> <sh...@gmail.com>wrote:
>>
>>>
>>>
>>>
>>> On Thu, Dec 20, 2012 at 4:52 AM, Koushik Das
>>> <ko...@citrix.com>wrote:
>>>
>>>> See inline
>>>>
>>>> Thanks,
>>>> Koushik
>>>>
>>>>> -----Original Message-----
>>>>> From: Chip Childers [mailto:chip.childers@sungard.com]
>>>>> Sent: Wednesday, December 19, 2012 7:55 PM
>>>>> To: cloudstack-dev@incubator.apache.org
>>>>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>>>
>>>>> On Wed, Dec 19, 2012 at 3:34 AM, Koushik Das
>>>>> <ko...@citrix.com>
>>>>> wrote:
>>>>>> See inline
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>>>> Sent: Tuesday, December 18, 2012 10:35 PM
>>>>>>> To: cloudstack-dev@incubator.apache.org
>>>>>>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>>>>>
>>>>>>> The FS looks good and addresses the things I'd want it to
>>>>>>> (scaling should be limited to within cluster, use offerings).
>>>>>>>
>>>>>>> As you mention, there's a real problem surrounding no support
>>>>>>> for scaling down CPU, and it's just as much a problem with the
>>>>>>> guests as it is with hvms at the moment, it seems. This makes it
>>>>>>> hard to just set a VM as a dynamic one, since at some point
>>>>>>> you'll likely trigger it to scale up and have to reboot to get
>>>>>>> back down. My suggestion if this goes through however is that
>>>>>>> instead of marking a vm for auto scale, we can either attach
>>>>>>> multiple compute offerings (with a priority or "level") to a VM,
>>>>>>> along with triggers (we can't really
>>>> trigger on
>>>>> memory, but perhaps cpu utilization over a specific time, e.g.
>>>>>>> if cpu is at 80% for x time, fall back to the next offering), or
>>>>>>> we can create a specific single compute offering that allows you
>>>>>>> to specify a min and max memory, cpu, and a trigger at which it
>>>>>>> scales (this latter one is my preference).
>>>>>>>
>>>>>>> The whole thing is problematic though, because people can
>>>>>>> inadvertently trigger their VM to scale up when they're
>>>>>>> installing updates or compiling or something and then have to
>>>>>>> reboot to come back down. If we can't take away resources
>>>>>>> without manual intervention, we shouldn't add them. For this
>>>>>>> reason I'd like to see the focus (at least initially) on simply
>>>>>>> being able to change to larger compute offerings while the VM is
>>>>>>> up. With this in place, if someone really wants to autoscale,
>>>>>>> they can use the api in a combination of fetching the VM stats
>>>>>>> and the existing changeServiceForVirtualMachine. Or we can put
>>>>>>> that in, but I think
>>>> any
>>>>> implementation will be a poor experience without being able to go
>>>>> both ways.
>>>>>>>
>>>>>>
>>>>>> This is a good suggestion but as you have mentioned first
>>>>>> priority is
>>>> to have
>>>>> the basic stuff working (increasing CPU/RAM for running VMs).
>>>>>> Also another thing is that HVs (at least Vmware) require that a
>>>>>> VM is
>>>>> configured appropriately when it is stopped in order to support
>>>> increasing
>>>>> CPU/RAM while it is running. We can either do this for all VMs
>>>> irrespective of
>>>>> the fact whether the CPU/RAM is going to be actually increased OR
>>>>> do it
>>>> only
>>>>> for selective VMs (maybe based on compute offering). If this is
>>>>> going
>>>> to be
>>>>> common across all HVs the latter can be done.
>>>>
>>>
>>> I think it could be done either way. The straightforward way is via
>>> offering that allows for max/current CPU and max/current RAM to be
>>> entered (basically exposing how the hypervisor settings themselves
>>> work). But you could also do a global setting of some sort that says
>>> 'set everything to a max of X CPU and Y RAM', so that every service
>>> offering can be upgraded live. As you mention, it will require at
>>> least a restart of the VMs to apply, so perhaps users could just
>>> switch service offerings anyway. It could be handy to allow people to
>>> upgrade service offering when it was unplanned for, though.
>>>
>>>
>>>>>>
>>>>>>> I don't know, maybe I'm off in left field here, I'd be
>>>>>>> interested in hearing the thoughts of others.
>>>>>>>
>>>>>>> You mention  'upgradeVirtualMachine', which should be mentioned
>>>>>>> on the customer facing API is called
>>>>>>> 'changeServiceForVirtualMachine', just to reduce confusion.
>>>>>>>
>>>>>>
>>>>>> upgradeVirtualMachine is an existing command (see
>>>>> UpgradeVMCmd.java), was planning to reuse it. But yes if the name
>>>>> sounds confusing we can deprecate it and create a new command with
>>>>> the name you have suggested.
>>>>>>
>>>>>
>>>>> Please don't break backward compatibility without the whole list
>>>> discussing
>>>>> the implications on a dedicated thread.  We had previously agreed
>>>>> that
>>>> we
>>>>> were going to maintain API compatibility between 4.0.0-incubating
>>>>> and
>>>> our
>>>>> next feature release.  If we break it, we have to release as
>>>> 5.0.0-incubating
>>>>> instead of 4.1.0-incubating.
>>>>
>>>> In that case will add a new async API changeServiceForVirtualMachine
>>>> (or if anyone else comes up with a better name) which will work for
>>>> both running and stopped VMs. upgradeVirtualMachine would continue to
>>>> exist till
>>>> 5.0.0 happens.
>>>>
>>>
>>> Would this break backward compatibility? If an API call goes from
>>> upgrading VMs only while they're off, and still upgrades VMs only
>>> while they're off, but also upgrades VMs with a newer, specific
>>> service offering type while they're on, does that break backward
>>> compatibility? Or let's say we simply removed the check to make sure
>>> the VM was off, and instead just checked if the VM was started with
>>> the newer compatible settings... would that break backward
>>> compatibility? The call still does what it did before when used as
>>> before (changes service offering while the VM is off).
>>>
>>> Regarding upgradeVirtualMachine, I saw no mention of it in the API
>>> docs, and found that in the code, changeServiceForVirtualMachine was
>>> mapped to UpgradeVMCmd.java, which is why I mentioned the confusion.
>>> 'upgradeVirtualMachine' only exists as an internal method of the
>>> userVmService. See the file "client/tomcatconf/commands.properties.in"
>>>
>>> changeServiceForVirtualMachine=com.cloud.api.commands.UpgradeVMCmd
>>>
>>>
>>>
>>>>
>>>>>
>>>>>>>
>>>>>>> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das
>>>>>>> <koushik.das@citrix.com
>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Created first draft of the FS
>>>>>>>>
>>>>>>>
>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scal
>>>>> in
>>>>>>> g
>>>>>>>> +of+CPU+and+RAM
>>>>>>>> Also created jira issue
>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-658
>>>>>>>>
>>>>>>>> Comments? There is an 'open issue' section where I have
>>>>>>>> mentioned some issues that needs to be closed
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Koushik
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Koushik Das [mailto:koushik.das@citrix.com]
>>>>>>>>> Sent: Saturday, December 15, 2012 11:14 PM
>>>>>>>>> To: cloudstack-dev@incubator.apache.org
>>>>>>>>> Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>>>>>>>
>>>>>>>>> Currently CS supports changing CPU and RAM for stopped VM.
>>>>>>>>> This is achieved by changing compute offering of the VM
>>>>>>>>> (with new CPU and RAM
>>>>>>>>> values) and then starting it. I am planning to extend the
>>>>>>>>> same for
>>>>>>>> running VM
>>>>>>>>> as well. Initially planning to do it for Vmware where CPU
>>>>>>>>> and RAM can be dynamically increased. Support of other HVs
>>>>>>>>> can also be added if they support increasing CPU/RAM.
>>>>>>>>>
>>>>>>>>> Assuming that in the updated compute offering only CPU and
>>>>>>>>> RAM has changed, the deployment planner can either select
>>>>>>>>> the same host in which case the values are dynamically
>>>>>>>>> scaled up OR a different one in which
>>>>>>>> case
>>>>>>>>> the operation fails. In future if there is support for live
>>>>>>>>> migration
>>>>>>>> (provided
>>>>>>>>> HV supports it) then another option in the latter case could
>>>>>>>>> be to
>>>>>>>> migrate the
>>>>>>>>> VM first and then scale it up.
>>>>>>>>>
>>>>>>>>> I will start working on the FS and share it out sometime
>>>>>>>>> next
>>>> week.
>>>>>>>>>
>>>>>>>>> Comments/suggestions?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Koushik
>>>>>>>>
>>>>>>
>>>>
>>>
>>>
>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Nitin Mehta <Ni...@citrix.com>]

Hari - I am yet to update the FS. I will do that very soon. Answer to your
questions inline.

On 22/01/13 10:50 AM, "Hari Kannan" <ha...@citrix.com> wrote:

>Hello Nitin, Koushik,
>
>I'm following up on this feature - is the FS located here still
>accurate/up to date?
>
>I also wish to get clarification on a couple of things:
>
>1.	There is a reference - open issue 1: "Ability to mark the VM for scale
>up at creation time " - what is the intent behind this capability? Why
>cant every VM be capable of scaling? Also, given the capability of
>scaling up is actually a property of {OS, Hypervisor} what would be the
>intent behind having this as a property of a service offering? How was
>this "closed"?

We would not be doing this. The intent earlier was because Vmware required
this property to be set upfront (while deploying vm itself). We will set
this property for every vm be default.

>2.	We also know that XS and KVM support this for Linux (max needs to be
>pre-defined) - so, I assume we are supporting both these platforms, in
>addition to VMware?

I am not planning to do this. I think for KVM Marcus was going to pitch in.

>3.	In case there is no capacity in cluster to scale up, just making sure
>that the existing VM will not have any impact, right?

Yes, there won't be any impact. Don't worry about it :)

>
>Hari
>
>-----Original Message-----
>From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>Sent: Thursday, December 20, 2012 9:47 AM
>To: cloudstack-dev@incubator.apache.org
>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>
>Oh, if it's not already obvious, we're onboard for collaborating on this
>feature and can help implement the KVM hypervisor portions. :-)
>
>
>On Thu, Dec 20, 2012 at 8:44 AM, Marcus Sorensen
><sh...@gmail.com>wrote:
>
>>
>>
>>
>> On Thu, Dec 20, 2012 at 4:52 AM, Koushik Das
>><ko...@citrix.com>wrote:
>>
>>> See inline
>>>
>>> Thanks,
>>> Koushik
>>>
>>> > -----Original Message-----
>>> > From: Chip Childers [mailto:chip.childers@sungard.com]
>>> > Sent: Wednesday, December 19, 2012 7:55 PM
>>> > To: cloudstack-dev@incubator.apache.org
>>> > Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> >
>>> > On Wed, Dec 19, 2012 at 3:34 AM, Koushik Das
>>> > <ko...@citrix.com>
>>> > wrote:
>>> > > See inline
>>> > >
>>> > >> -----Original Message-----
>>> > >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>> > >> Sent: Tuesday, December 18, 2012 10:35 PM
>>> > >> To: cloudstack-dev@incubator.apache.org
>>> > >> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> > >>
>>> > >> The FS looks good and addresses the things I'd want it to
>>> > >> (scaling should be limited to within cluster, use offerings).
>>> > >>
>>> > >> As you mention, there's a real problem surrounding no support
>>> > >> for scaling down CPU, and it's just as much a problem with the
>>> > >> guests as it is with hvms at the moment, it seems. This makes it
>>> > >> hard to just set a VM as a dynamic one, since at some point
>>> > >> you'll likely trigger it to scale up and have to reboot to get
>>> > >> back down. My suggestion if this goes through however is that
>>> > >> instead of marking a vm for auto scale, we can either attach
>>> > >> multiple compute offerings (with a priority or "level") to a VM,
>>> > >> along with triggers (we can't really
>>> trigger on
>>> > memory, but perhaps cpu utilization over a specific time, e.g.
>>> > >> if cpu is at 80% for x time, fall back to the next offering), or
>>> > >> we can create a specific single compute offering that allows you
>>> > >> to specify a min and max memory, cpu, and a trigger at which it
>>> > >> scales (this latter one is my preference).
>>> > >>
>>> > >> The whole thing is problematic though, because people can
>>> > >> inadvertently trigger their VM to scale up when they're
>>> > >> installing updates or compiling or something and then have to
>>> > >> reboot to come back down. If we can't take away resources
>>> > >> without manual intervention, we shouldn't add them. For this
>>> > >> reason I'd like to see the focus (at least initially) on simply
>>> > >> being able to change to larger compute offerings while the VM is
>>> > >> up. With this in place, if someone really wants to autoscale,
>>> > >> they can use the api in a combination of fetching the VM stats
>>> > >> and the existing changeServiceForVirtualMachine. Or we can put
>>> > >> that in, but I think
>>> any
>>> > implementation will be a poor experience without being able to go
>>> > both ways.
>>> > >>
>>> > >
>>> > > This is a good suggestion but as you have mentioned first
>>> > > priority is
>>> to have
>>> > the basic stuff working (increasing CPU/RAM for running VMs).
>>> > > Also another thing is that HVs (at least Vmware) require that a
>>> > > VM is
>>> > configured appropriately when it is stopped in order to support
>>> increasing
>>> > CPU/RAM while it is running. We can either do this for all VMs
>>> irrespective of
>>> > the fact whether the CPU/RAM is going to be actually increased OR
>>> > do it
>>> only
>>> > for selective VMs (maybe based on compute offering). If this is
>>> > going
>>> to be
>>> > common across all HVs the latter can be done.
>>>
>>
>> I think it could be done either way. The straightforward way is via
>> offering that allows for max/current CPU and max/current RAM to be
>> entered (basically exposing how the hypervisor settings themselves
>> work). But you could also do a global setting of some sort that says
>> 'set everything to a max of X CPU and Y RAM', so that every service
>> offering can be upgraded live. As you mention, it will require at
>> least a restart of the VMs to apply, so perhaps users could just
>> switch service offerings anyway. It could be handy to allow people to
>> upgrade service offering when it was unplanned for, though.
>>
>>
>>> > >
>>> > >> I don't know, maybe I'm off in left field here, I'd be
>>> > >> interested in hearing the thoughts of others.
>>> > >>
>>> > >> You mention  'upgradeVirtualMachine', which should be mentioned
>>> > >> on the customer facing API is called
>>> > >> 'changeServiceForVirtualMachine', just to reduce confusion.
>>> > >>
>>> > >
>>> > > upgradeVirtualMachine is an existing command (see
>>> > UpgradeVMCmd.java), was planning to reuse it. But yes if the name
>>> > sounds confusing we can deprecate it and create a new command with
>>> > the name you have suggested.
>>> > >
>>> >
>>> > Please don't break backward compatibility without the whole list
>>> discussing
>>> > the implications on a dedicated thread.  We had previously agreed
>>> > that
>>> we
>>> > were going to maintain API compatibility between 4.0.0-incubating
>>> > and
>>> our
>>> > next feature release.  If we break it, we have to release as
>>> 5.0.0-incubating
>>> > instead of 4.1.0-incubating.
>>>
>>> In that case will add a new async API changeServiceForVirtualMachine
>>> (or if anyone else comes up with a better name) which will work for
>>> both running and stopped VMs. upgradeVirtualMachine would continue to
>>> exist till
>>> 5.0.0 happens.
>>>
>>
>> Would this break backward compatibility? If an API call goes from
>> upgrading VMs only while they're off, and still upgrades VMs only
>> while they're off, but also upgrades VMs with a newer, specific
>> service offering type while they're on, does that break backward
>> compatibility? Or let's say we simply removed the check to make sure
>> the VM was off, and instead just checked if the VM was started with
>> the newer compatible settings... would that break backward
>> compatibility? The call still does what it did before when used as
>>before (changes service offering while the VM is off).
>>
>> Regarding upgradeVirtualMachine, I saw no mention of it in the API
>> docs, and found that in the code, changeServiceForVirtualMachine was
>> mapped to UpgradeVMCmd.java, which is why I mentioned the confusion.
>> 'upgradeVirtualMachine' only exists as an internal method of the
>> userVmService. See the file "client/tomcatconf/commands.properties.in"
>>
>> changeServiceForVirtualMachine=com.cloud.api.commands.UpgradeVMCmd
>>
>>
>>
>>>
>>> >
>>> > >>
>>> > >> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das
>>> > >> <koushik.das@citrix.com
>>> >
>>> > >> wrote:
>>> > >>
>>> > >> > Created first draft of the FS
>>> > >> >
>>> > >>
>>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scal
>>> > in
>>> > >> g
>>> > >> > +of+CPU+and+RAM
>>> > >> > Also created jira issue
>>> > >> > https://issues.apache.org/jira/browse/CLOUDSTACK-658
>>> > >> >
>>> > >> > Comments? There is an 'open issue' section where I have
>>> > >> > mentioned some issues that needs to be closed
>>> > >> >
>>> > >> > Thanks,
>>> > >> > Koushik
>>> > >> >
>>> > >> > > -----Original Message-----
>>> > >> > > From: Koushik Das [mailto:koushik.das@citrix.com]
>>> > >> > > Sent: Saturday, December 15, 2012 11:14 PM
>>> > >> > > To: cloudstack-dev@incubator.apache.org
>>> > >> > > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> > >> > >
>>> > >> > > Currently CS supports changing CPU and RAM for stopped VM.
>>> > >> > > This is achieved by changing compute offering of the VM
>>> > >> > > (with new CPU and RAM
>>> > >> > > values) and then starting it. I am planning to extend the
>>> > >> > > same for
>>> > >> > running VM
>>> > >> > > as well. Initially planning to do it for Vmware where CPU
>>> > >> > > and RAM can be dynamically increased. Support of other HVs
>>> > >> > > can also be added if they support increasing CPU/RAM.
>>> > >> > >
>>> > >> > > Assuming that in the updated compute offering only CPU and
>>> > >> > > RAM has changed, the deployment planner can either select
>>> > >> > > the same host in which case the values are dynamically
>>> > >> > > scaled up OR a different one in which
>>> > >> > case
>>> > >> > > the operation fails. In future if there is support for live
>>> > >> > > migration
>>> > >> > (provided
>>> > >> > > HV supports it) then another option in the latter case could
>>> > >> > > be to
>>> > >> > migrate the
>>> > >> > > VM first and then scale it up.
>>> > >> > >
>>> > >> > > I will start working on the FS and share it out sometime
>>> > >> > > next
>>> week.
>>> > >> > >
>>> > >> > > Comments/suggestions?
>>> > >> > >
>>> > >> > > Thanks,
>>> > >> > > Koushik
>>> > >> >
>>> > >
>>>
>>
>>

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Hari Kannan <ha...@citrix.com>]

Hello Nitin, Koushik,

I'm following up on this feature - is the FS located here still accurate/up to date?

I also wish to get clarification on a couple of things:

1.	There is a reference - open issue 1: "Ability to mark the VM for scale up at creation time " - what is the intent behind this capability? Why cant every VM be capable of scaling? Also, given the capability of scaling up is actually a property of {OS, Hypervisor} what would be the intent behind having this as a property of a service offering? How was this "closed"?
2.	We also know that XS and KVM support this for Linux (max needs to be pre-defined) - so, I assume we are supporting both these platforms, in addition to VMware?
3.	In case there is no capacity in cluster to scale up, just making sure that the existing VM will not have any impact, right?

Hari

-----Original Message-----
From: Marcus Sorensen [mailto:shadowsor@gmail.com] 
Sent: Thursday, December 20, 2012 9:47 AM
To: cloudstack-dev@incubator.apache.org
Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs

Oh, if it's not already obvious, we're onboard for collaborating on this feature and can help implement the KVM hypervisor portions. :-)


On Thu, Dec 20, 2012 at 8:44 AM, Marcus Sorensen <sh...@gmail.com>wrote:

>
>
>
> On Thu, Dec 20, 2012 at 4:52 AM, Koushik Das <ko...@citrix.com>wrote:
>
>> See inline
>>
>> Thanks,
>> Koushik
>>
>> > -----Original Message-----
>> > From: Chip Childers [mailto:chip.childers@sungard.com]
>> > Sent: Wednesday, December 19, 2012 7:55 PM
>> > To: cloudstack-dev@incubator.apache.org
>> > Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>> >
>> > On Wed, Dec 19, 2012 at 3:34 AM, Koushik Das 
>> > <ko...@citrix.com>
>> > wrote:
>> > > See inline
>> > >
>> > >> -----Original Message-----
>> > >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> > >> Sent: Tuesday, December 18, 2012 10:35 PM
>> > >> To: cloudstack-dev@incubator.apache.org
>> > >> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>> > >>
>> > >> The FS looks good and addresses the things I'd want it to 
>> > >> (scaling should be limited to within cluster, use offerings).
>> > >>
>> > >> As you mention, there's a real problem surrounding no support 
>> > >> for scaling down CPU, and it's just as much a problem with the 
>> > >> guests as it is with hvms at the moment, it seems. This makes it 
>> > >> hard to just set a VM as a dynamic one, since at some point 
>> > >> you'll likely trigger it to scale up and have to reboot to get 
>> > >> back down. My suggestion if this goes through however is that 
>> > >> instead of marking a vm for auto scale, we can either attach 
>> > >> multiple compute offerings (with a priority or "level") to a VM, 
>> > >> along with triggers (we can't really
>> trigger on
>> > memory, but perhaps cpu utilization over a specific time, e.g.
>> > >> if cpu is at 80% for x time, fall back to the next offering), or 
>> > >> we can create a specific single compute offering that allows you 
>> > >> to specify a min and max memory, cpu, and a trigger at which it 
>> > >> scales (this latter one is my preference).
>> > >>
>> > >> The whole thing is problematic though, because people can 
>> > >> inadvertently trigger their VM to scale up when they're 
>> > >> installing updates or compiling or something and then have to 
>> > >> reboot to come back down. If we can't take away resources 
>> > >> without manual intervention, we shouldn't add them. For this 
>> > >> reason I'd like to see the focus (at least initially) on simply 
>> > >> being able to change to larger compute offerings while the VM is 
>> > >> up. With this in place, if someone really wants to autoscale, 
>> > >> they can use the api in a combination of fetching the VM stats 
>> > >> and the existing changeServiceForVirtualMachine. Or we can put 
>> > >> that in, but I think
>> any
>> > implementation will be a poor experience without being able to go 
>> > both ways.
>> > >>
>> > >
>> > > This is a good suggestion but as you have mentioned first 
>> > > priority is
>> to have
>> > the basic stuff working (increasing CPU/RAM for running VMs).
>> > > Also another thing is that HVs (at least Vmware) require that a 
>> > > VM is
>> > configured appropriately when it is stopped in order to support
>> increasing
>> > CPU/RAM while it is running. We can either do this for all VMs
>> irrespective of
>> > the fact whether the CPU/RAM is going to be actually increased OR 
>> > do it
>> only
>> > for selective VMs (maybe based on compute offering). If this is 
>> > going
>> to be
>> > common across all HVs the latter can be done.
>>
>
> I think it could be done either way. The straightforward way is via 
> offering that allows for max/current CPU and max/current RAM to be 
> entered (basically exposing how the hypervisor settings themselves 
> work). But you could also do a global setting of some sort that says 
> 'set everything to a max of X CPU and Y RAM', so that every service 
> offering can be upgraded live. As you mention, it will require at 
> least a restart of the VMs to apply, so perhaps users could just 
> switch service offerings anyway. It could be handy to allow people to 
> upgrade service offering when it was unplanned for, though.
>
>
>> > >
>> > >> I don't know, maybe I'm off in left field here, I'd be 
>> > >> interested in hearing the thoughts of others.
>> > >>
>> > >> You mention  'upgradeVirtualMachine', which should be mentioned 
>> > >> on the customer facing API is called 
>> > >> 'changeServiceForVirtualMachine', just to reduce confusion.
>> > >>
>> > >
>> > > upgradeVirtualMachine is an existing command (see
>> > UpgradeVMCmd.java), was planning to reuse it. But yes if the name 
>> > sounds confusing we can deprecate it and create a new command with 
>> > the name you have suggested.
>> > >
>> >
>> > Please don't break backward compatibility without the whole list
>> discussing
>> > the implications on a dedicated thread.  We had previously agreed 
>> > that
>> we
>> > were going to maintain API compatibility between 4.0.0-incubating 
>> > and
>> our
>> > next feature release.  If we break it, we have to release as
>> 5.0.0-incubating
>> > instead of 4.1.0-incubating.
>>
>> In that case will add a new async API changeServiceForVirtualMachine 
>> (or if anyone else comes up with a better name) which will work for 
>> both running and stopped VMs. upgradeVirtualMachine would continue to 
>> exist till
>> 5.0.0 happens.
>>
>
> Would this break backward compatibility? If an API call goes from 
> upgrading VMs only while they're off, and still upgrades VMs only 
> while they're off, but also upgrades VMs with a newer, specific 
> service offering type while they're on, does that break backward 
> compatibility? Or let's say we simply removed the check to make sure 
> the VM was off, and instead just checked if the VM was started with 
> the newer compatible settings... would that break backward 
> compatibility? The call still does what it did before when used as before (changes service offering while the VM is off).
>
> Regarding upgradeVirtualMachine, I saw no mention of it in the API 
> docs, and found that in the code, changeServiceForVirtualMachine was 
> mapped to UpgradeVMCmd.java, which is why I mentioned the confusion.
> 'upgradeVirtualMachine' only exists as an internal method of the 
> userVmService. See the file "client/tomcatconf/commands.properties.in"
>
> changeServiceForVirtualMachine=com.cloud.api.commands.UpgradeVMCmd
>
>
>
>>
>> >
>> > >>
>> > >> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das 
>> > >> <koushik.das@citrix.com
>> >
>> > >> wrote:
>> > >>
>> > >> > Created first draft of the FS
>> > >> >
>> > >>
>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scal
>> > in
>> > >> g
>> > >> > +of+CPU+and+RAM
>> > >> > Also created jira issue
>> > >> > https://issues.apache.org/jira/browse/CLOUDSTACK-658
>> > >> >
>> > >> > Comments? There is an 'open issue' section where I have 
>> > >> > mentioned some issues that needs to be closed
>> > >> >
>> > >> > Thanks,
>> > >> > Koushik
>> > >> >
>> > >> > > -----Original Message-----
>> > >> > > From: Koushik Das [mailto:koushik.das@citrix.com]
>> > >> > > Sent: Saturday, December 15, 2012 11:14 PM
>> > >> > > To: cloudstack-dev@incubator.apache.org
>> > >> > > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>> > >> > >
>> > >> > > Currently CS supports changing CPU and RAM for stopped VM. 
>> > >> > > This is achieved by changing compute offering of the VM 
>> > >> > > (with new CPU and RAM
>> > >> > > values) and then starting it. I am planning to extend the 
>> > >> > > same for
>> > >> > running VM
>> > >> > > as well. Initially planning to do it for Vmware where CPU 
>> > >> > > and RAM can be dynamically increased. Support of other HVs 
>> > >> > > can also be added if they support increasing CPU/RAM.
>> > >> > >
>> > >> > > Assuming that in the updated compute offering only CPU and 
>> > >> > > RAM has changed, the deployment planner can either select 
>> > >> > > the same host in which case the values are dynamically 
>> > >> > > scaled up OR a different one in which
>> > >> > case
>> > >> > > the operation fails. In future if there is support for live 
>> > >> > > migration
>> > >> > (provided
>> > >> > > HV supports it) then another option in the latter case could 
>> > >> > > be to
>> > >> > migrate the
>> > >> > > VM first and then scale it up.
>> > >> > >
>> > >> > > I will start working on the FS and share it out sometime 
>> > >> > > next
>> week.
>> > >> > >
>> > >> > > Comments/suggestions?
>> > >> > >
>> > >> > > Thanks,
>> > >> > > Koushik
>> > >> >
>> > >
>>
>
>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Marcus Sorensen <sh...@gmail.com>]

Oh, if it's not already obvious, we're onboard for collaborating on this
feature and can help implement the KVM hypervisor portions. :-)


On Thu, Dec 20, 2012 at 8:44 AM, Marcus Sorensen <sh...@gmail.com>wrote:

>
>
>
> On Thu, Dec 20, 2012 at 4:52 AM, Koushik Das <ko...@citrix.com>wrote:
>
>> See inline
>>
>> Thanks,
>> Koushik
>>
>> > -----Original Message-----
>> > From: Chip Childers [mailto:chip.childers@sungard.com]
>> > Sent: Wednesday, December 19, 2012 7:55 PM
>> > To: cloudstack-dev@incubator.apache.org
>> > Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>> >
>> > On Wed, Dec 19, 2012 at 3:34 AM, Koushik Das <ko...@citrix.com>
>> > wrote:
>> > > See inline
>> > >
>> > >> -----Original Message-----
>> > >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> > >> Sent: Tuesday, December 18, 2012 10:35 PM
>> > >> To: cloudstack-dev@incubator.apache.org
>> > >> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>> > >>
>> > >> The FS looks good and addresses the things I'd want it to (scaling
>> > >> should be limited to within cluster, use offerings).
>> > >>
>> > >> As you mention, there's a real problem surrounding no support for
>> > >> scaling down CPU, and it's just as much a problem with the guests as
>> > >> it is with hvms at the moment, it seems. This makes it hard to just
>> > >> set a VM as a dynamic one, since at some point you'll likely trigger
>> > >> it to scale up and have to reboot to get back down. My suggestion if
>> > >> this goes through however is that instead of marking a vm for auto
>> > >> scale, we can either attach multiple compute offerings (with a
>> > >> priority or "level") to a VM, along with triggers (we can't really
>> trigger on
>> > memory, but perhaps cpu utilization over a specific time, e.g.
>> > >> if cpu is at 80% for x time, fall back to the next offering), or we
>> > >> can create a specific single compute offering that allows you to
>> > >> specify a min and max memory, cpu, and a trigger at which it scales
>> > >> (this latter one is my preference).
>> > >>
>> > >> The whole thing is problematic though, because people can
>> > >> inadvertently trigger their VM to scale up when they're installing
>> > >> updates or compiling or something and then have to reboot to come
>> > >> back down. If we can't take away resources without manual
>> > >> intervention, we shouldn't add them. For this reason I'd like to see
>> > >> the focus (at least initially) on simply being able to change to
>> > >> larger compute offerings while the VM is up. With this in place, if
>> > >> someone really wants to autoscale, they can use the api in a
>> > >> combination of fetching the VM stats and the existing
>> > >> changeServiceForVirtualMachine. Or we can put that in, but I think
>> any
>> > implementation will be a poor experience without being able to go both
>> > ways.
>> > >>
>> > >
>> > > This is a good suggestion but as you have mentioned first priority is
>> to have
>> > the basic stuff working (increasing CPU/RAM for running VMs).
>> > > Also another thing is that HVs (at least Vmware) require that a VM is
>> > configured appropriately when it is stopped in order to support
>> increasing
>> > CPU/RAM while it is running. We can either do this for all VMs
>> irrespective of
>> > the fact whether the CPU/RAM is going to be actually increased OR do it
>> only
>> > for selective VMs (maybe based on compute offering). If this is going
>> to be
>> > common across all HVs the latter can be done.
>>
>
> I think it could be done either way. The straightforward way is via
> offering that allows for max/current CPU and max/current RAM to be entered
> (basically exposing how the hypervisor settings themselves work). But you
> could also do a global setting of some sort that says 'set everything to a
> max of X CPU and Y RAM', so that every service offering can be upgraded
> live. As you mention, it will require at least a restart of the VMs to
> apply, so perhaps users could just switch service offerings anyway. It
> could be handy to allow people to upgrade service offering when it was
> unplanned for, though.
>
>
>> > >
>> > >> I don't know, maybe I'm off in left field here, I'd be interested in
>> > >> hearing the thoughts of others.
>> > >>
>> > >> You mention  'upgradeVirtualMachine', which should be mentioned on
>> > >> the customer facing API is called 'changeServiceForVirtualMachine',
>> > >> just to reduce confusion.
>> > >>
>> > >
>> > > upgradeVirtualMachine is an existing command (see
>> > UpgradeVMCmd.java), was planning to reuse it. But yes if the name sounds
>> > confusing we can deprecate it and create a new command with the name
>> > you have suggested.
>> > >
>> >
>> > Please don't break backward compatibility without the whole list
>> discussing
>> > the implications on a dedicated thread.  We had previously agreed that
>> we
>> > were going to maintain API compatibility between 4.0.0-incubating and
>> our
>> > next feature release.  If we break it, we have to release as
>> 5.0.0-incubating
>> > instead of 4.1.0-incubating.
>>
>> In that case will add a new async API changeServiceForVirtualMachine (or
>> if anyone else comes up with a better name) which will work for both
>> running and stopped VMs. upgradeVirtualMachine would continue to exist till
>> 5.0.0 happens.
>>
>
> Would this break backward compatibility? If an API call goes from
> upgrading VMs only while they're off, and still upgrades VMs only while
> they're off, but also upgrades VMs with a newer, specific service offering
> type while they're on, does that break backward compatibility? Or let's say
> we simply removed the check to make sure the VM was off, and instead just
> checked if the VM was started with the newer compatible settings... would
> that break backward compatibility? The call still does what it did before
> when used as before (changes service offering while the VM is off).
>
> Regarding upgradeVirtualMachine, I saw no mention of it in the API docs,
> and found that in the code, changeServiceForVirtualMachine was mapped to
> UpgradeVMCmd.java, which is why I mentioned the confusion.
> 'upgradeVirtualMachine' only exists as an internal method of the
> userVmService. See the file "client/tomcatconf/commands.properties.in"
>
> changeServiceForVirtualMachine=com.cloud.api.commands.UpgradeVMCmd
>
>
>
>>
>> >
>> > >>
>> > >> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das <koushik.das@citrix.com
>> >
>> > >> wrote:
>> > >>
>> > >> > Created first draft of the FS
>> > >> >
>> > >>
>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scalin
>> > >> g
>> > >> > +of+CPU+and+RAM
>> > >> > Also created jira issue
>> > >> > https://issues.apache.org/jira/browse/CLOUDSTACK-658
>> > >> >
>> > >> > Comments? There is an 'open issue' section where I have mentioned
>> > >> > some issues that needs to be closed
>> > >> >
>> > >> > Thanks,
>> > >> > Koushik
>> > >> >
>> > >> > > -----Original Message-----
>> > >> > > From: Koushik Das [mailto:koushik.das@citrix.com]
>> > >> > > Sent: Saturday, December 15, 2012 11:14 PM
>> > >> > > To: cloudstack-dev@incubator.apache.org
>> > >> > > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>> > >> > >
>> > >> > > Currently CS supports changing CPU and RAM for stopped VM. This
>> > >> > > is achieved by changing compute offering of the VM (with new CPU
>> > >> > > and RAM
>> > >> > > values) and then starting it. I am planning to extend the same
>> > >> > > for
>> > >> > running VM
>> > >> > > as well. Initially planning to do it for Vmware where CPU and RAM
>> > >> > > can be dynamically increased. Support of other HVs can also be
>> > >> > > added if they support increasing CPU/RAM.
>> > >> > >
>> > >> > > Assuming that in the updated compute offering only CPU and RAM
>> > >> > > has changed, the deployment planner can either select the same
>> > >> > > host in which case the values are dynamically scaled up OR a
>> > >> > > different one in which
>> > >> > case
>> > >> > > the operation fails. In future if there is support for live
>> > >> > > migration
>> > >> > (provided
>> > >> > > HV supports it) then another option in the latter case could be
>> > >> > > to
>> > >> > migrate the
>> > >> > > VM first and then scale it up.
>> > >> > >
>> > >> > > I will start working on the FS and share it out sometime next
>> week.
>> > >> > >
>> > >> > > Comments/suggestions?
>> > >> > >
>> > >> > > Thanks,
>> > >> > > Koushik
>> > >> >
>> > >
>>
>
>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Marcus Sorensen <sh...@gmail.com>]

On Thu, Dec 20, 2012 at 4:52 AM, Koushik Das <ko...@citrix.com> wrote:

> See inline
>
> Thanks,
> Koushik
>
> > -----Original Message-----
> > From: Chip Childers [mailto:chip.childers@sungard.com]
> > Sent: Wednesday, December 19, 2012 7:55 PM
> > To: cloudstack-dev@incubator.apache.org
> > Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> >
> > On Wed, Dec 19, 2012 at 3:34 AM, Koushik Das <ko...@citrix.com>
> > wrote:
> > > See inline
> > >
> > >> -----Original Message-----
> > >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> > >> Sent: Tuesday, December 18, 2012 10:35 PM
> > >> To: cloudstack-dev@incubator.apache.org
> > >> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> > >>
> > >> The FS looks good and addresses the things I'd want it to (scaling
> > >> should be limited to within cluster, use offerings).
> > >>
> > >> As you mention, there's a real problem surrounding no support for
> > >> scaling down CPU, and it's just as much a problem with the guests as
> > >> it is with hvms at the moment, it seems. This makes it hard to just
> > >> set a VM as a dynamic one, since at some point you'll likely trigger
> > >> it to scale up and have to reboot to get back down. My suggestion if
> > >> this goes through however is that instead of marking a vm for auto
> > >> scale, we can either attach multiple compute offerings (with a
> > >> priority or "level") to a VM, along with triggers (we can't really
> trigger on
> > memory, but perhaps cpu utilization over a specific time, e.g.
> > >> if cpu is at 80% for x time, fall back to the next offering), or we
> > >> can create a specific single compute offering that allows you to
> > >> specify a min and max memory, cpu, and a trigger at which it scales
> > >> (this latter one is my preference).
> > >>
> > >> The whole thing is problematic though, because people can
> > >> inadvertently trigger their VM to scale up when they're installing
> > >> updates or compiling or something and then have to reboot to come
> > >> back down. If we can't take away resources without manual
> > >> intervention, we shouldn't add them. For this reason I'd like to see
> > >> the focus (at least initially) on simply being able to change to
> > >> larger compute offerings while the VM is up. With this in place, if
> > >> someone really wants to autoscale, they can use the api in a
> > >> combination of fetching the VM stats and the existing
> > >> changeServiceForVirtualMachine. Or we can put that in, but I think any
> > implementation will be a poor experience without being able to go both
> > ways.
> > >>
> > >
> > > This is a good suggestion but as you have mentioned first priority is
> to have
> > the basic stuff working (increasing CPU/RAM for running VMs).
> > > Also another thing is that HVs (at least Vmware) require that a VM is
> > configured appropriately when it is stopped in order to support
> increasing
> > CPU/RAM while it is running. We can either do this for all VMs
> irrespective of
> > the fact whether the CPU/RAM is going to be actually increased OR do it
> only
> > for selective VMs (maybe based on compute offering). If this is going to
> be
> > common across all HVs the latter can be done.
>

I think it could be done either way. The straightforward way is via
offering that allows for max/current CPU and max/current RAM to be entered
(basically exposing how the hypervisor settings themselves work). But you
could also do a global setting of some sort that says 'set everything to a
max of X CPU and Y RAM', so that every service offering can be upgraded
live. As you mention, it will require at least a restart of the VMs to
apply, so perhaps users could just switch service offerings anyway. It
could be handy to allow people to upgrade service offering when it was
unplanned for, though.


> > >
> > >> I don't know, maybe I'm off in left field here, I'd be interested in
> > >> hearing the thoughts of others.
> > >>
> > >> You mention  'upgradeVirtualMachine', which should be mentioned on
> > >> the customer facing API is called 'changeServiceForVirtualMachine',
> > >> just to reduce confusion.
> > >>
> > >
> > > upgradeVirtualMachine is an existing command (see
> > UpgradeVMCmd.java), was planning to reuse it. But yes if the name sounds
> > confusing we can deprecate it and create a new command with the name
> > you have suggested.
> > >
> >
> > Please don't break backward compatibility without the whole list
> discussing
> > the implications on a dedicated thread.  We had previously agreed that we
> > were going to maintain API compatibility between 4.0.0-incubating and our
> > next feature release.  If we break it, we have to release as
> 5.0.0-incubating
> > instead of 4.1.0-incubating.
>
> In that case will add a new async API changeServiceForVirtualMachine (or
> if anyone else comes up with a better name) which will work for both
> running and stopped VMs. upgradeVirtualMachine would continue to exist till
> 5.0.0 happens.
>

Would this break backward compatibility? If an API call goes from upgrading
VMs only while they're off, and still upgrades VMs only while they're off,
but also upgrades VMs with a newer, specific service offering type while
they're on, does that break backward compatibility? Or let's say we simply
removed the check to make sure the VM was off, and instead just checked if
the VM was started with the newer compatible settings... would that break
backward compatibility? The call still does what it did before when used as
before (changes service offering while the VM is off).

Regarding upgradeVirtualMachine, I saw no mention of it in the API docs,
and found that in the code, changeServiceForVirtualMachine was mapped to
UpgradeVMCmd.java, which is why I mentioned the confusion.
'upgradeVirtualMachine' only exists as an internal method of the
userVmService. See the file "client/tomcatconf/commands.properties.in"

changeServiceForVirtualMachine=com.cloud.api.commands.UpgradeVMCmd



>
> >
> > >>
> > >> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das <ko...@citrix.com>
> > >> wrote:
> > >>
> > >> > Created first draft of the FS
> > >> >
> > >>
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scalin
> > >> g
> > >> > +of+CPU+and+RAM
> > >> > Also created jira issue
> > >> > https://issues.apache.org/jira/browse/CLOUDSTACK-658
> > >> >
> > >> > Comments? There is an 'open issue' section where I have mentioned
> > >> > some issues that needs to be closed
> > >> >
> > >> > Thanks,
> > >> > Koushik
> > >> >
> > >> > > -----Original Message-----
> > >> > > From: Koushik Das [mailto:koushik.das@citrix.com]
> > >> > > Sent: Saturday, December 15, 2012 11:14 PM
> > >> > > To: cloudstack-dev@incubator.apache.org
> > >> > > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> > >> > >
> > >> > > Currently CS supports changing CPU and RAM for stopped VM. This
> > >> > > is achieved by changing compute offering of the VM (with new CPU
> > >> > > and RAM
> > >> > > values) and then starting it. I am planning to extend the same
> > >> > > for
> > >> > running VM
> > >> > > as well. Initially planning to do it for Vmware where CPU and RAM
> > >> > > can be dynamically increased. Support of other HVs can also be
> > >> > > added if they support increasing CPU/RAM.
> > >> > >
> > >> > > Assuming that in the updated compute offering only CPU and RAM
> > >> > > has changed, the deployment planner can either select the same
> > >> > > host in which case the values are dynamically scaled up OR a
> > >> > > different one in which
> > >> > case
> > >> > > the operation fails. In future if there is support for live
> > >> > > migration
> > >> > (provided
> > >> > > HV supports it) then another option in the latter case could be
> > >> > > to
> > >> > migrate the
> > >> > > VM first and then scale it up.
> > >> > >
> > >> > > I will start working on the FS and share it out sometime next
> week.
> > >> > >
> > >> > > Comments/suggestions?
> > >> > >
> > >> > > Thanks,
> > >> > > Koushik
> > >> >
> > >
>

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Koushik Das <ko...@citrix.com>]

See inline

Thanks,
Koushik

> -----Original Message-----
> From: Chip Childers [mailto:chip.childers@sungard.com]
> Sent: Wednesday, December 19, 2012 7:55 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> 
> On Wed, Dec 19, 2012 at 3:34 AM, Koushik Das <ko...@citrix.com>
> wrote:
> > See inline
> >
> >> -----Original Message-----
> >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> >> Sent: Tuesday, December 18, 2012 10:35 PM
> >> To: cloudstack-dev@incubator.apache.org
> >> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>
> >> The FS looks good and addresses the things I'd want it to (scaling
> >> should be limited to within cluster, use offerings).
> >>
> >> As you mention, there's a real problem surrounding no support for
> >> scaling down CPU, and it's just as much a problem with the guests as
> >> it is with hvms at the moment, it seems. This makes it hard to just
> >> set a VM as a dynamic one, since at some point you'll likely trigger
> >> it to scale up and have to reboot to get back down. My suggestion if
> >> this goes through however is that instead of marking a vm for auto
> >> scale, we can either attach multiple compute offerings (with a
> >> priority or "level") to a VM, along with triggers (we can't really trigger on
> memory, but perhaps cpu utilization over a specific time, e.g.
> >> if cpu is at 80% for x time, fall back to the next offering), or we
> >> can create a specific single compute offering that allows you to
> >> specify a min and max memory, cpu, and a trigger at which it scales
> >> (this latter one is my preference).
> >>
> >> The whole thing is problematic though, because people can
> >> inadvertently trigger their VM to scale up when they're installing
> >> updates or compiling or something and then have to reboot to come
> >> back down. If we can't take away resources without manual
> >> intervention, we shouldn't add them. For this reason I'd like to see
> >> the focus (at least initially) on simply being able to change to
> >> larger compute offerings while the VM is up. With this in place, if
> >> someone really wants to autoscale, they can use the api in a
> >> combination of fetching the VM stats and the existing
> >> changeServiceForVirtualMachine. Or we can put that in, but I think any
> implementation will be a poor experience without being able to go both
> ways.
> >>
> >
> > This is a good suggestion but as you have mentioned first priority is to have
> the basic stuff working (increasing CPU/RAM for running VMs).
> > Also another thing is that HVs (at least Vmware) require that a VM is
> configured appropriately when it is stopped in order to support increasing
> CPU/RAM while it is running. We can either do this for all VMs irrespective of
> the fact whether the CPU/RAM is going to be actually increased OR do it only
> for selective VMs (maybe based on compute offering). If this is going to be
> common across all HVs the latter can be done.
> >
> >> I don't know, maybe I'm off in left field here, I'd be interested in
> >> hearing the thoughts of others.
> >>
> >> You mention  'upgradeVirtualMachine', which should be mentioned on
> >> the customer facing API is called 'changeServiceForVirtualMachine',
> >> just to reduce confusion.
> >>
> >
> > upgradeVirtualMachine is an existing command (see
> UpgradeVMCmd.java), was planning to reuse it. But yes if the name sounds
> confusing we can deprecate it and create a new command with the name
> you have suggested.
> >
> 
> Please don't break backward compatibility without the whole list discussing
> the implications on a dedicated thread.  We had previously agreed that we
> were going to maintain API compatibility between 4.0.0-incubating and our
> next feature release.  If we break it, we have to release as 5.0.0-incubating
> instead of 4.1.0-incubating.

In that case will add a new async API changeServiceForVirtualMachine (or if anyone else comes up with a better name) which will work for both running and stopped VMs. upgradeVirtualMachine would continue to exist till 5.0.0 happens.

> 
> >>
> >> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das <ko...@citrix.com>
> >> wrote:
> >>
> >> > Created first draft of the FS
> >> >
> >>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scalin
> >> g
> >> > +of+CPU+and+RAM
> >> > Also created jira issue
> >> > https://issues.apache.org/jira/browse/CLOUDSTACK-658
> >> >
> >> > Comments? There is an 'open issue' section where I have mentioned
> >> > some issues that needs to be closed
> >> >
> >> > Thanks,
> >> > Koushik
> >> >
> >> > > -----Original Message-----
> >> > > From: Koushik Das [mailto:koushik.das@citrix.com]
> >> > > Sent: Saturday, December 15, 2012 11:14 PM
> >> > > To: cloudstack-dev@incubator.apache.org
> >> > > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> >> > >
> >> > > Currently CS supports changing CPU and RAM for stopped VM. This
> >> > > is achieved by changing compute offering of the VM (with new CPU
> >> > > and RAM
> >> > > values) and then starting it. I am planning to extend the same
> >> > > for
> >> > running VM
> >> > > as well. Initially planning to do it for Vmware where CPU and RAM
> >> > > can be dynamically increased. Support of other HVs can also be
> >> > > added if they support increasing CPU/RAM.
> >> > >
> >> > > Assuming that in the updated compute offering only CPU and RAM
> >> > > has changed, the deployment planner can either select the same
> >> > > host in which case the values are dynamically scaled up OR a
> >> > > different one in which
> >> > case
> >> > > the operation fails. In future if there is support for live
> >> > > migration
> >> > (provided
> >> > > HV supports it) then another option in the latter case could be
> >> > > to
> >> > migrate the
> >> > > VM first and then scale it up.
> >> > >
> >> > > I will start working on the FS and share it out sometime next week.
> >> > >
> >> > > Comments/suggestions?
> >> > >
> >> > > Thanks,
> >> > > Koushik
> >> >
> >

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Chip Childers <ch...@sungard.com>]

On Wed, Dec 19, 2012 at 3:34 AM, Koushik Das <ko...@citrix.com> wrote:
> See inline
>
>> -----Original Message-----
>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> Sent: Tuesday, December 18, 2012 10:35 PM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>
>> The FS looks good and addresses the things I'd want it to (scaling should be
>> limited to within cluster, use offerings).
>>
>> As you mention, there's a real problem surrounding no support for scaling
>> down CPU, and it's just as much a problem with the guests as it is with hvms
>> at the moment, it seems. This makes it hard to just set a VM as a dynamic
>> one, since at some point you'll likely trigger it to scale up and have to reboot
>> to get back down. My suggestion if this goes through however is that instead
>> of marking a vm for auto scale, we can either attach multiple compute
>> offerings (with a priority or "level") to a VM, along with triggers (we can't
>> really trigger on memory, but perhaps cpu utilization over a specific time, e.g.
>> if cpu is at 80% for x time, fall back to the next offering), or we can create a
>> specific single compute offering that allows you to specify a min and max
>> memory, cpu, and a trigger at which it scales (this latter one is my
>> preference).
>>
>> The whole thing is problematic though, because people can inadvertently
>> trigger their VM to scale up when they're installing updates or compiling or
>> something and then have to reboot to come back down. If we can't take
>> away resources without manual intervention, we shouldn't add them. For
>> this reason I'd like to see the focus (at least initially) on simply being able to
>> change to larger compute offerings while the VM is up. With this in place, if
>> someone really wants to autoscale, they can use the api in a combination of
>> fetching the VM stats and the existing changeServiceForVirtualMachine. Or
>> we can put that in, but I think any implementation will be a poor experience
>> without being able to go both ways.
>>
>
> This is a good suggestion but as you have mentioned first priority is to have the basic stuff working (increasing CPU/RAM for running VMs).
> Also another thing is that HVs (at least Vmware) require that a VM is configured appropriately when it is stopped in order to support increasing CPU/RAM while it is running. We can either do this for all VMs irrespective of the fact whether the CPU/RAM is going to be actually increased OR do it only for selective VMs (maybe based on compute offering). If this is going to be common across all HVs the latter can be done.
>
>> I don't know, maybe I'm off in left field here, I'd be interested in hearing the
>> thoughts of others.
>>
>> You mention  'upgradeVirtualMachine', which should be mentioned on the
>> customer facing API is called 'changeServiceForVirtualMachine', just to
>> reduce confusion.
>>
>
> upgradeVirtualMachine is an existing command (see UpgradeVMCmd.java), was planning to reuse it. But yes if the name sounds confusing we can deprecate it and create a new command with the name you have suggested.
>

Please don't break backward compatibility without the whole list
discussing the implications on a dedicated thread.  We had previously
agreed that we were going to maintain API compatibility between
4.0.0-incubating and our next feature release.  If we break it, we
have to release as 5.0.0-incubating instead of 4.1.0-incubating.

>>
>> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das <ko...@citrix.com>
>> wrote:
>>
>> > Created first draft of the FS
>> >
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scaling
>> > +of+CPU+and+RAM
>> > Also created jira issue
>> > https://issues.apache.org/jira/browse/CLOUDSTACK-658
>> >
>> > Comments? There is an 'open issue' section where I have mentioned some
>> > issues that needs to be closed
>> >
>> > Thanks,
>> > Koushik
>> >
>> > > -----Original Message-----
>> > > From: Koushik Das [mailto:koushik.das@citrix.com]
>> > > Sent: Saturday, December 15, 2012 11:14 PM
>> > > To: cloudstack-dev@incubator.apache.org
>> > > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>> > >
>> > > Currently CS supports changing CPU and RAM for stopped VM. This is
>> > > achieved by changing compute offering of the VM (with new CPU and
>> > > RAM
>> > > values) and then starting it. I am planning to extend the same for
>> > running VM
>> > > as well. Initially planning to do it for Vmware where CPU and RAM
>> > > can be dynamically increased. Support of other HVs can also be added
>> > > if they support increasing CPU/RAM.
>> > >
>> > > Assuming that in the updated compute offering only CPU and RAM has
>> > > changed, the deployment planner can either select the same host in
>> > > which case the values are dynamically scaled up OR a different one
>> > > in which
>> > case
>> > > the operation fails. In future if there is support for live
>> > > migration
>> > (provided
>> > > HV supports it) then another option in the latter case could be to
>> > migrate the
>> > > VM first and then scale it up.
>> > >
>> > > I will start working on the FS and share it out sometime next week.
>> > >
>> > > Comments/suggestions?
>> > >
>> > > Thanks,
>> > > Koushik
>> >
>

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Koushik Das <ko...@citrix.com>]

See inline

> -----Original Message-----
> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> Sent: Tuesday, December 18, 2012 10:35 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> 
> The FS looks good and addresses the things I'd want it to (scaling should be
> limited to within cluster, use offerings).
> 
> As you mention, there's a real problem surrounding no support for scaling
> down CPU, and it's just as much a problem with the guests as it is with hvms
> at the moment, it seems. This makes it hard to just set a VM as a dynamic
> one, since at some point you'll likely trigger it to scale up and have to reboot
> to get back down. My suggestion if this goes through however is that instead
> of marking a vm for auto scale, we can either attach multiple compute
> offerings (with a priority or "level") to a VM, along with triggers (we can't
> really trigger on memory, but perhaps cpu utilization over a specific time, e.g.
> if cpu is at 80% for x time, fall back to the next offering), or we can create a
> specific single compute offering that allows you to specify a min and max
> memory, cpu, and a trigger at which it scales (this latter one is my
> preference).
> 
> The whole thing is problematic though, because people can inadvertently
> trigger their VM to scale up when they're installing updates or compiling or
> something and then have to reboot to come back down. If we can't take
> away resources without manual intervention, we shouldn't add them. For
> this reason I'd like to see the focus (at least initially) on simply being able to
> change to larger compute offerings while the VM is up. With this in place, if
> someone really wants to autoscale, they can use the api in a combination of
> fetching the VM stats and the existing changeServiceForVirtualMachine. Or
> we can put that in, but I think any implementation will be a poor experience
> without being able to go both ways.
> 

This is a good suggestion but as you have mentioned first priority is to have the basic stuff working (increasing CPU/RAM for running VMs).
Also another thing is that HVs (at least Vmware) require that a VM is configured appropriately when it is stopped in order to support increasing CPU/RAM while it is running. We can either do this for all VMs irrespective of the fact whether the CPU/RAM is going to be actually increased OR do it only for selective VMs (maybe based on compute offering). If this is going to be common across all HVs the latter can be done.

> I don't know, maybe I'm off in left field here, I'd be interested in hearing the
> thoughts of others.
> 
> You mention  'upgradeVirtualMachine', which should be mentioned on the
> customer facing API is called 'changeServiceForVirtualMachine', just to
> reduce confusion.
> 

upgradeVirtualMachine is an existing command (see UpgradeVMCmd.java), was planning to reuse it. But yes if the name sounds confusing we can deprecate it and create a new command with the name you have suggested.

> 
> On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das <ko...@citrix.com>
> wrote:
> 
> > Created first draft of the FS
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scaling
> > +of+CPU+and+RAM
> > Also created jira issue
> > https://issues.apache.org/jira/browse/CLOUDSTACK-658
> >
> > Comments? There is an 'open issue' section where I have mentioned some
> > issues that needs to be closed
> >
> > Thanks,
> > Koushik
> >
> > > -----Original Message-----
> > > From: Koushik Das [mailto:koushik.das@citrix.com]
> > > Sent: Saturday, December 15, 2012 11:14 PM
> > > To: cloudstack-dev@incubator.apache.org
> > > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> > >
> > > Currently CS supports changing CPU and RAM for stopped VM. This is
> > > achieved by changing compute offering of the VM (with new CPU and
> > > RAM
> > > values) and then starting it. I am planning to extend the same for
> > running VM
> > > as well. Initially planning to do it for Vmware where CPU and RAM
> > > can be dynamically increased. Support of other HVs can also be added
> > > if they support increasing CPU/RAM.
> > >
> > > Assuming that in the updated compute offering only CPU and RAM has
> > > changed, the deployment planner can either select the same host in
> > > which case the values are dynamically scaled up OR a different one
> > > in which
> > case
> > > the operation fails. In future if there is support for live
> > > migration
> > (provided
> > > HV supports it) then another option in the latter case could be to
> > migrate the
> > > VM first and then scale it up.
> > >
> > > I will start working on the FS and share it out sometime next week.
> > >
> > > Comments/suggestions?
> > >
> > > Thanks,
> > > Koushik
> >

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Marcus Sorensen <sh...@gmail.com>]

The FS looks good and addresses the things I'd want it to (scaling should
be limited to within cluster, use offerings).

As you mention, there's a real problem surrounding no support for scaling
down CPU, and it's just as much a problem with the guests as it is with
hvms at the moment, it seems. This makes it hard to just set a VM as a
dynamic one, since at some point you'll likely trigger it to scale up and
have to reboot to get back down. My suggestion if this goes
through however is that instead of marking a vm for auto scale, we can
either attach multiple compute offerings (with a priority or "level") to a
VM, along with triggers (we can't really trigger on memory, but perhaps cpu
utilization over a specific time, e.g. if cpu is at 80% for x time, fall
back to the next offering), or we can create a specific single compute
offering that allows you to specify a min and max memory, cpu, and a
trigger at which it scales (this latter one is my preference).

The whole thing is problematic though, because people can inadvertently
trigger their VM to scale up when they're installing updates or compiling
or something and then have to reboot to come back down. If we can't take
away resources without manual intervention, we shouldn't add them. For this
reason I'd like to see the focus (at least initially) on simply being able
to change to larger compute offerings while the VM is up. With this in
place, if someone really wants to autoscale, they can use the api in a
combination of fetching the VM stats and the existing
changeServiceForVirtualMachine. Or we can put that in, but I think any
implementation will be a poor experience without being able to go both ways.

I don't know, maybe I'm off in left field here, I'd be interested in
hearing the thoughts of others.

You mention  'upgradeVirtualMachine', which should be mentioned on the
customer facing API is called 'changeServiceForVirtualMachine', just to
reduce confusion.


On Tue, Dec 18, 2012 at 9:18 AM, Koushik Das <ko...@citrix.com> wrote:

> Created first draft of the FS
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scaling+of+CPU+and+RAM
> Also created jira issue
> https://issues.apache.org/jira/browse/CLOUDSTACK-658
>
> Comments? There is an 'open issue' section where I have mentioned some
> issues that needs to be closed
>
> Thanks,
> Koushik
>
> > -----Original Message-----
> > From: Koushik Das [mailto:koushik.das@citrix.com]
> > Sent: Saturday, December 15, 2012 11:14 PM
> > To: cloudstack-dev@incubator.apache.org
> > Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> >
> > Currently CS supports changing CPU and RAM for stopped VM. This is
> > achieved by changing compute offering of the VM (with new CPU and RAM
> > values) and then starting it. I am planning to extend the same for
> running VM
> > as well. Initially planning to do it for Vmware where CPU and RAM can be
> > dynamically increased. Support of other HVs can also be added if they
> > support increasing CPU/RAM.
> >
> > Assuming that in the updated compute offering only CPU and RAM has
> > changed, the deployment planner can either select the same host in which
> > case the values are dynamically scaled up OR a different one in which
> case
> > the operation fails. In future if there is support for live migration
> (provided
> > HV supports it) then another option in the latter case could be to
> migrate the
> > VM first and then scale it up.
> >
> > I will start working on the FS and share it out sometime next week.
> >
> > Comments/suggestions?
> >
> > Thanks,
> > Koushik
>

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Koushik Das <ko...@citrix.com>]

Created first draft of the FS https://cwiki.apache.org/confluence/display/CLOUDSTACK/Dynamic+scaling+of+CPU+and+RAM
Also created jira issue https://issues.apache.org/jira/browse/CLOUDSTACK-658

Comments? There is an 'open issue' section where I have mentioned some issues that needs to be closed

Thanks,
Koushik

> -----Original Message-----
> From: Koushik Das [mailto:koushik.das@citrix.com]
> Sent: Saturday, December 15, 2012 11:14 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> 
> Currently CS supports changing CPU and RAM for stopped VM. This is
> achieved by changing compute offering of the VM (with new CPU and RAM
> values) and then starting it. I am planning to extend the same for running VM
> as well. Initially planning to do it for Vmware where CPU and RAM can be
> dynamically increased. Support of other HVs can also be added if they
> support increasing CPU/RAM.
> 
> Assuming that in the updated compute offering only CPU and RAM has
> changed, the deployment planner can either select the same host in which
> case the values are dynamically scaled up OR a different one in which case
> the operation fails. In future if there is support for live migration (provided
> HV supports it) then another option in the latter case could be to migrate the
> VM first and then scale it up.
> 
> I will start working on the FS and share it out sometime next week.
> 
> Comments/suggestions?
> 
> Thanks,
> Koushik

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Mice Xia <we...@gmail.com>]

Nitin & Kelven,

I haven't updated the codes with syncAsyncJobExecution, and the conclusion
is making all vm api synchronized? I'm ok with this change, currently, for
a single vm instance, few apis can work on it concurrently.

-Mice


On Fri, Mar 15, 2013 at 12:20 PM, Nitin Mehta <Ni...@citrix.com>wrote:

> Thanks Kelven.
> I saw that, but making this change would mean that all the vm API's -
> deployvm, start-stop, reboot, destroy vm etc.. will now be synchronized
> with this change.
>
> Hope you guys are ok with this ??
>
> Thanks,
> -Nitin
>
> On 13/03/13 11:31 PM, "Kelven Yang" <ke...@citrix.com> wrote:
>
> >Nitin,
> >
> >Sorry to reply late, have been busy working on a patch release for
> >customer.
> >
> >AsyncJob manager does provide a mechanism that you can do synchronized the
> >job execution against a object. You may check out
> >AsyncJobManager.syncAsyncJobExecution().
> >
> >Kelven
> >
> >On 3/12/13 11:27 AM, "Nitin Mehta" <Ni...@citrix.com> wrote:
> >>syncAsyncJobExecutionThanks Alex.
> >>
> >>Kelven / Alex - I took a look into this and it seems that the framework
> >>has the capability to do this and this would solve the problem for
> >>scaling
> >>up vm and vm snapshots.
> >>But, we currently don't do synchronization for the vm object. Doing this
> >>means all the vm operations will be syncronized now. Are we fine with
> >>that
> >>?
> >>
> >>Thanks,
> >>-Nitin
> >>
> >>On 09/03/13 10:36 PM, "Alex Huang" <Al...@citrix.com> wrote:
> >>
> >>>Nitin,
> >>>
> >>>The other approach to this is to utilize the syncing feature in the job
> >>>queue.  I've cced Kelven to see if he can give you more detail.  His
> >>>code
> >>>is capable of syncing operations on a single object so you don't have to
> >>>add processing states.
> >>>
> >>>Given that all of your operations and vm snapshot operations must have
> >>>come in through the job queue, you might already have that ability to
> >>>not
> >>>interfere with each other.
> >>>
> >>>VM States are different because there can be outside changes (through
> >>>other vm managers) that cause vm life cycle to behave differently.
> >>>
> >>>--Alex
> >>>
> >>>> -----Original Message-----
> >>>> From: Nitin Mehta
> >>>> Sent: Saturday, March 9, 2013 2:35 AM
> >>>> To: cloudstack-dev@incubator.apache.org; Prashant Kumar Mishra;
> >>>> Abhinandan Prateek; Alex Huang
> >>>> Cc: Chip Childers
> >>>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>>
> >>>> Hi Alex,
> >>>> I had one more question. Say the MS is shut down or restarted, when do
> >>>>we
> >>>> clear this attribute in this case ?
> >>>>
> >>>> On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
> >>>>
> >>>> >Alex - Thanks very much for pointing out earlier this week that for
> >>>> >scaling up the vm we shouldn't change the vm lifecycle. I also read
> >>>> >http://markmail.org/message/6c6njactsklot62h
> >>>> >and understand that scaling up a vm is a vm operation and shouldn't
> >>>>be
> >>>> >mixed with vm lifecycle. So as you suggested in the thread that if I
> >>>> >need to prevent other Vm operations happening during this operation I
> >>>> >would need to introduce an attribute
> >>>> >
> >>>> >1. For this I would need to introduce a column in vm_instance table
> >>>> >which would be set during scale up operation.
> >>>> >2. To prevent other operations from happening this attribute needs to
> >>>> >be checked in all the other vm operations. There is no single common
> >>>> >piece of code where I can put the check so I have to explicitly check
> >>>> >for this attribute in all the operations code right ? I see that for
> >>>>"vm
> >>>> snapshot"
> >>>> >operation we have put this check in vm state transition method but
> >>>>this
> >>>> >method is called only for vm lifecycle changes. So when "vm snapshot"
> >>>> >happens the user might also scale up the vm. There might be a need
> >>>>for
> >>>> >them to be exclusive.
> >>>> >3. If I need to say lock capacity before the operation and modify it
> >>>> >after the operation is done (say during failure) how do I do it w/o
> >>>> >coupling the code changes or is it ok for now to do so ?
> >>>> >
> >>>> >
> >>>> >Thanks,
> >>>> >-Nitin
> >>>> >
> >>>> >On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
> >>>> >
> >>>> >>Hi Nitin,
> >>>> >>
> >>>> >>Please see below
> >>>> >>
> >>>> >>Hari
> >>>> >>
> >>>> >>-----Original Message-----
> >>>> >>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
> >>>> >>Sent: Tuesday, February 12, 2013 7:15 AM
> >>>> >>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
> >>>> >>Abhinandan Prateek
> >>>> >>Cc: Chip Childers
> >>>> >>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>> >>
> >>>> >>Apologize for the delayed response. Was involved in other issues.
> >>>> >>Please find answers inline.
> >>>> >>
> >>>> >>>
> >>>> >>>-----Original Message-----
> >>>> >>>From: Prashant Kumar Mishra
> >>>>[mailto:prashantkumar.mishra@citrix.com]
> >>>> >>>Sent: Thursday, January 24, 2013 12:26 PM
> >>>> >>>To: cloudstack-dev@incubator.apache.org
> >>>> >>>Cc: Nitin Mehta
> >>>> >>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>> >>>
> >>>> >>>Hi Nitin,
> >>>> >>>I am planning to take the QA job for this feature. Have reviewed
> >>>>the
> >>>> >>>functional spec, gone through community discussion  and have the
> >>>> >>>following questions
> >>>> >>>
> >>>> >>>1-What is expected behavior of CS for Operating systems which do
> >>>>not
> >>>> >>>support dynamic scaling . ?
> >>>> >>
> >>>> >>Will throw a not supported exception
> >>>> >>
> >>>> >>Hari: How do we know which OS is supported or not? Is it going to be
> >>>> >>part of the "capabilities" of hypervisor? Or where will this be
> >>>> >>specified/configured?
> >>>> >>PS: I know we plan to implement this only on VMware for now, but
> >>>>when
> >>>> >>installed/shipped, how will CS know the supported Hypervisor/OS?
> >>>> >>
> >>>> >>>
> >>>> >>>2-How much resources can be scaled up, is it limited by
> >>>>availability
> >>>> >>>of resource on host .?
> >>>> >>>
> >>>> >>>[Koushik Das ]
> >>>> >>>"Having a range for CPU/RAM in compute offering is definitely
> >>>>another
> >>>> >>>way of doing it. But creating the higher limit would be tricky. I
> >>>>am
> >>>> >>>not sure if it is always known to users how much they want to scale
> >>>> >>>up to at the time of deploying VM. Moreover if the higher limit is
> >>>> >>>known then the VM can be deployed with that value itself. Also in
> >>>> >>>case of having a range in the offering the usage part needs to be
> >>>> >>>handled appropriately. Currently usage is purely based on the
> >>>> >>>offering and individual values are not stored".
> >>>> >>>[/Koushik Das]
> >>>> >>
> >>>> >>It is not limited by the resources on host but on the available
> >>>> >>capacity in any of the host within the cluster.
> >>>> >>
> >>>> >>Hari: I'm not sure I understand the question - how is this any
> >>>> >>different than requesting a new VM? Or upgrading from offering A to
> >>>> >>Offering B that exists today (although VM needs to be shutdown)?
> >>>> >>
> >>>> >>>
> >>>> >>>it seems  its totally depend on service offering , please correct
> >>>>me
> >>>> >>>if I am wrong.
> >>>> >>>
> >>>> >>>3-  Scheduled snapshot of volumes during the operation .
> >>>> >>>
> >>>> >>>[NITIN]
> >>>> >>>For vmware, the entire vm is locked by HV and this can be an issue.
> >>>>I
> >>>> >>>will leverage on current implementations for existing interactions
> >>>> >>>like scheduled snapshots events during live migration and will
> >>>> >>>replicate the same.
> >>>> >>>[/NITIN]
> >>>> >>>
> >>>> >>>Can you elaborate what is expected in case of VMware .
> >>>> >>
> >>>> >>What I mean is there is an existing functionality which is
> >>>>implemented
> >>>> >>the same way. I will just do it the same way.
> >>>> >>
> >>>> >>>
> >>>> >>>4 - what is expected behavior in case of  powers off the vm during
> >>>> >>>the operation .? is it different for different hypervisors.?
> >>>> >>
> >>>> >>There is nothing much to do for powered off vms because we will just
> >>>> >>update the DB. When the vm is started it will pick up these values
> >>>> >>from the DB.
> >>>> >>This functionality already exists.
> >>>> >>
> >>>> >>>
> >>>> >>>5- what is expected in case of migration fails( In FS no
> >>>>description
> >>>> >>>about this),
> >>>> >>>       -CS will  retry to migrate it again if yes how many time ?
> >>>> >>>      - will it mark as a failure and can't  scale up(even
> >>>>resources
> >>>> >>>are available in cluster) ?
> >>>> >>
> >>>> >>We will retry N (configurable) times and if unsuccessful we will
> >>>>throw
> >>>> >>an exception to the user.
> >>>> >>
> >>>> >>Hari: Can you please elaborate why a migration might fail? And, is
> >>>>the
> >>>> >>"N" configurable times a new global?
> >>>> >>
> >>>> >>>
> >>>> >>>6- Apart from  "scaleVirtualMachine"    any other APIs are getting
> >>>> >>>changed ?
> >>>> >>
> >>>> >>No
> >>>> >>
> >>>> >>>
> >>>> >>>7-Scale down is allowed ? (still open issue in FS)
> >>>> >>
> >>>> >>No for time being.
> >>>> >>
> >>>> >>>
> >>>> >>>8-Are we going to introduce custom compute offering (still open
> >>>>issue
> >>>> >>>in
> >>>> >>>FS) ?
> >>>> >>
> >>>> >>No for now
> >>>> >>
> >>>> >>>
> >>>> >>>9- what are the guide line for upgrade  ?
> >>>> >>
> >>>> >>There is nothing for upgrade because we do not introduce new values
> >>>>in
> >>>> >>DB.
> >>>> >>
> >>>> >>>
> >>>> >>>10-Any DB changes ?
> >>>> >>
> >>>> >>See #9 above.
> >>>> >>
> >>>> >>>
> >>>> >>>11- which Usage events are getting introduced for billing .?
> >>>> >>
> >>>> >>Will update the FS.
> >>>> >>
> >>>> >>>
> >>>> >>>12-hypervisor support ,is it only for VMware (as per FS)  or its
> >>>> >>>getting extended for XS/KVM also ?
> >>>> >>
> >>>> >>There are subtasks opened for XS and KVM. I am doing it only for
> >>>>Vmware.
> >>>> >>
> >>>> >>>
> >>>> >>>Thanks
> >>>> >>>Prashant Kumar Mishra
> >>>> >>>
> >>>> >>>
> >>>> >>>-----Original Message-----
> >>>> >>>From: Koushik Das [mailto:koushik.das@citrix.com]
> >>>> >>>Sent: Saturday, December 15, 2012 11:14 PM
> >>>> >>>To: cloudstack-dev@incubator.apache.org
> >>>> >>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>> >>>
> >>>> >>>Currently CS supports changing CPU and RAM for stopped VM. This is
> >>>> >>>achieved by changing compute offering of the VM (with new CPU and
> >>>> RAM
> >>>> >>>values) and then starting it. I am planning to extend the same for
> >>>> >>>running VM as well. Initially planning to do it for Vmware where
> >>>>CPU
> >>>> >>>and RAM can be dynamically increased. Support of other HVs can also
> >>>> >>>be added if they support increasing CPU/RAM.
> >>>> >>>
> >>>> >>>Assuming that in the updated compute offering only CPU and RAM has
> >>>> >>>changed, the deployment planner can either select the same host in
> >>>> >>>which case the values are dynamically scaled up OR a different one
> >>>>in
> >>>> >>>which case the operation fails. In future if there is support for
> >>>> >>>live migration (provided HV supports it) then another option in the
> >>>> >>>latter case could be to migrate the VM first and then scale it up.
> >>>> >>>
> >>>> >>>I will start working on the FS and share it out sometime next week.
> >>>> >>>
> >>>> >>>Comments/suggestions?
> >>>> >>>
> >>>> >>>Thanks,
> >>>> >>>Koushik
> >>>> >>
> >>>> >
> >>>
> >>
> >
>
>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Nitin Mehta <Ni...@citrix.com>]

Thanks Kelven.
I saw that, but making this change would mean that all the vm API's -
deployvm, start-stop, reboot, destroy vm etc.. will now be synchronized
with this change.

Hope you guys are ok with this ??

Thanks,
-Nitin

On 13/03/13 11:31 PM, "Kelven Yang" <ke...@citrix.com> wrote:

>Nitin,
>
>Sorry to reply late, have been busy working on a patch release for
>customer. 
>
>AsyncJob manager does provide a mechanism that you can do synchronized the
>job execution against a object. You may check out
>AsyncJobManager.syncAsyncJobExecution().
>
>Kelven
>
>On 3/12/13 11:27 AM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>>syncAsyncJobExecutionThanks Alex.
>>
>>Kelven / Alex - I took a look into this and it seems that the framework
>>has the capability to do this and this would solve the problem for
>>scaling
>>up vm and vm snapshots.
>>But, we currently don't do synchronization for the vm object. Doing this
>>means all the vm operations will be syncronized now. Are we fine with
>>that
>>?  
>>
>>Thanks,
>>-Nitin
>>
>>On 09/03/13 10:36 PM, "Alex Huang" <Al...@citrix.com> wrote:
>>
>>>Nitin,
>>>
>>>The other approach to this is to utilize the syncing feature in the job
>>>queue.  I've cced Kelven to see if he can give you more detail.  His
>>>code
>>>is capable of syncing operations on a single object so you don't have to
>>>add processing states.
>>>
>>>Given that all of your operations and vm snapshot operations must have
>>>come in through the job queue, you might already have that ability to
>>>not
>>>interfere with each other.
>>>
>>>VM States are different because there can be outside changes (through
>>>other vm managers) that cause vm life cycle to behave differently.
>>>
>>>--Alex
>>>
>>>> -----Original Message-----
>>>> From: Nitin Mehta
>>>> Sent: Saturday, March 9, 2013 2:35 AM
>>>> To: cloudstack-dev@incubator.apache.org; Prashant Kumar Mishra;
>>>> Abhinandan Prateek; Alex Huang
>>>> Cc: Chip Childers
>>>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>> 
>>>> Hi Alex,
>>>> I had one more question. Say the MS is shut down or restarted, when do
>>>>we
>>>> clear this attribute in this case ?
>>>> 
>>>> On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>>>> 
>>>> >Alex - Thanks very much for pointing out earlier this week that for
>>>> >scaling up the vm we shouldn't change the vm lifecycle. I also read
>>>> >http://markmail.org/message/6c6njactsklot62h
>>>> >and understand that scaling up a vm is a vm operation and shouldn't
>>>>be
>>>> >mixed with vm lifecycle. So as you suggested in the thread that if I
>>>> >need to prevent other Vm operations happening during this operation I
>>>> >would need to introduce an attribute
>>>> >
>>>> >1. For this I would need to introduce a column in vm_instance table
>>>> >which would be set during scale up operation.
>>>> >2. To prevent other operations from happening this attribute needs to
>>>> >be checked in all the other vm operations. There is no single common
>>>> >piece of code where I can put the check so I have to explicitly check
>>>> >for this attribute in all the operations code right ? I see that for
>>>>"vm
>>>> snapshot"
>>>> >operation we have put this check in vm state transition method but
>>>>this
>>>> >method is called only for vm lifecycle changes. So when "vm snapshot"
>>>> >happens the user might also scale up the vm. There might be a need
>>>>for
>>>> >them to be exclusive.
>>>> >3. If I need to say lock capacity before the operation and modify it
>>>> >after the operation is done (say during failure) how do I do it w/o
>>>> >coupling the code changes or is it ok for now to do so ?
>>>> >
>>>> >
>>>> >Thanks,
>>>> >-Nitin
>>>> >
>>>> >On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
>>>> >
>>>> >>Hi Nitin,
>>>> >>
>>>> >>Please see below
>>>> >>
>>>> >>Hari
>>>> >>
>>>> >>-----Original Message-----
>>>> >>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
>>>> >>Sent: Tuesday, February 12, 2013 7:15 AM
>>>> >>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
>>>> >>Abhinandan Prateek
>>>> >>Cc: Chip Childers
>>>> >>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>> >>
>>>> >>Apologize for the delayed response. Was involved in other issues.
>>>> >>Please find answers inline.
>>>> >>
>>>> >>>
>>>> >>>-----Original Message-----
>>>> >>>From: Prashant Kumar Mishra
>>>>[mailto:prashantkumar.mishra@citrix.com]
>>>> >>>Sent: Thursday, January 24, 2013 12:26 PM
>>>> >>>To: cloudstack-dev@incubator.apache.org
>>>> >>>Cc: Nitin Mehta
>>>> >>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>> >>>
>>>> >>>Hi Nitin,
>>>> >>>I am planning to take the QA job for this feature. Have reviewed
>>>>the
>>>> >>>functional spec, gone through community discussion  and have the
>>>> >>>following questions
>>>> >>>
>>>> >>>1-What is expected behavior of CS for Operating systems which do
>>>>not
>>>> >>>support dynamic scaling . ?
>>>> >>
>>>> >>Will throw a not supported exception
>>>> >>
>>>> >>Hari: How do we know which OS is supported or not? Is it going to be
>>>> >>part of the "capabilities" of hypervisor? Or where will this be
>>>> >>specified/configured?
>>>> >>PS: I know we plan to implement this only on VMware for now, but
>>>>when
>>>> >>installed/shipped, how will CS know the supported Hypervisor/OS?
>>>> >>
>>>> >>>
>>>> >>>2-How much resources can be scaled up, is it limited by
>>>>availability
>>>> >>>of resource on host .?
>>>> >>>
>>>> >>>[Koushik Das ]
>>>> >>>"Having a range for CPU/RAM in compute offering is definitely
>>>>another
>>>> >>>way of doing it. But creating the higher limit would be tricky. I
>>>>am
>>>> >>>not sure if it is always known to users how much they want to scale
>>>> >>>up to at the time of deploying VM. Moreover if the higher limit is
>>>> >>>known then the VM can be deployed with that value itself. Also in
>>>> >>>case of having a range in the offering the usage part needs to be
>>>> >>>handled appropriately. Currently usage is purely based on the
>>>> >>>offering and individual values are not stored".
>>>> >>>[/Koushik Das]
>>>> >>
>>>> >>It is not limited by the resources on host but on the available
>>>> >>capacity in any of the host within the cluster.
>>>> >>
>>>> >>Hari: I'm not sure I understand the question - how is this any
>>>> >>different than requesting a new VM? Or upgrading from offering A to
>>>> >>Offering B that exists today (although VM needs to be shutdown)?
>>>> >>
>>>> >>>
>>>> >>>it seems  its totally depend on service offering , please correct
>>>>me
>>>> >>>if I am wrong.
>>>> >>>
>>>> >>>3-  Scheduled snapshot of volumes during the operation .
>>>> >>>
>>>> >>>[NITIN]
>>>> >>>For vmware, the entire vm is locked by HV and this can be an issue.
>>>>I
>>>> >>>will leverage on current implementations for existing interactions
>>>> >>>like scheduled snapshots events during live migration and will
>>>> >>>replicate the same.
>>>> >>>[/NITIN]
>>>> >>>
>>>> >>>Can you elaborate what is expected in case of VMware .
>>>> >>
>>>> >>What I mean is there is an existing functionality which is
>>>>implemented
>>>> >>the same way. I will just do it the same way.
>>>> >>
>>>> >>>
>>>> >>>4 - what is expected behavior in case of  powers off the vm during
>>>> >>>the operation .? is it different for different hypervisors.?
>>>> >>
>>>> >>There is nothing much to do for powered off vms because we will just
>>>> >>update the DB. When the vm is started it will pick up these values
>>>> >>from the DB.
>>>> >>This functionality already exists.
>>>> >>
>>>> >>>
>>>> >>>5- what is expected in case of migration fails( In FS no
>>>>description
>>>> >>>about this),
>>>> >>>       -CS will  retry to migrate it again if yes how many time ?
>>>> >>>      - will it mark as a failure and can't  scale up(even
>>>>resources
>>>> >>>are available in cluster) ?
>>>> >>
>>>> >>We will retry N (configurable) times and if unsuccessful we will
>>>>throw
>>>> >>an exception to the user.
>>>> >>
>>>> >>Hari: Can you please elaborate why a migration might fail? And, is
>>>>the
>>>> >>"N" configurable times a new global?
>>>> >>
>>>> >>>
>>>> >>>6- Apart from  "scaleVirtualMachine"    any other APIs are getting
>>>> >>>changed ?
>>>> >>
>>>> >>No
>>>> >>
>>>> >>>
>>>> >>>7-Scale down is allowed ? (still open issue in FS)
>>>> >>
>>>> >>No for time being.
>>>> >>
>>>> >>>
>>>> >>>8-Are we going to introduce custom compute offering (still open
>>>>issue
>>>> >>>in
>>>> >>>FS) ?
>>>> >>
>>>> >>No for now
>>>> >>
>>>> >>>
>>>> >>>9- what are the guide line for upgrade  ?
>>>> >>
>>>> >>There is nothing for upgrade because we do not introduce new values
>>>>in
>>>> >>DB.
>>>> >>
>>>> >>>
>>>> >>>10-Any DB changes ?
>>>> >>
>>>> >>See #9 above.
>>>> >>
>>>> >>>
>>>> >>>11- which Usage events are getting introduced for billing .?
>>>> >>
>>>> >>Will update the FS.
>>>> >>
>>>> >>>
>>>> >>>12-hypervisor support ,is it only for VMware (as per FS)  or its
>>>> >>>getting extended for XS/KVM also ?
>>>> >>
>>>> >>There are subtasks opened for XS and KVM. I am doing it only for
>>>>Vmware.
>>>> >>
>>>> >>>
>>>> >>>Thanks
>>>> >>>Prashant Kumar Mishra
>>>> >>>
>>>> >>>
>>>> >>>-----Original Message-----
>>>> >>>From: Koushik Das [mailto:koushik.das@citrix.com]
>>>> >>>Sent: Saturday, December 15, 2012 11:14 PM
>>>> >>>To: cloudstack-dev@incubator.apache.org
>>>> >>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>> >>>
>>>> >>>Currently CS supports changing CPU and RAM for stopped VM. This is
>>>> >>>achieved by changing compute offering of the VM (with new CPU and
>>>> RAM
>>>> >>>values) and then starting it. I am planning to extend the same for
>>>> >>>running VM as well. Initially planning to do it for Vmware where
>>>>CPU
>>>> >>>and RAM can be dynamically increased. Support of other HVs can also
>>>> >>>be added if they support increasing CPU/RAM.
>>>> >>>
>>>> >>>Assuming that in the updated compute offering only CPU and RAM has
>>>> >>>changed, the deployment planner can either select the same host in
>>>> >>>which case the values are dynamically scaled up OR a different one
>>>>in
>>>> >>>which case the operation fails. In future if there is support for
>>>> >>>live migration (provided HV supports it) then another option in the
>>>> >>>latter case could be to migrate the VM first and then scale it up.
>>>> >>>
>>>> >>>I will start working on the FS and share it out sometime next week.
>>>> >>>
>>>> >>>Comments/suggestions?
>>>> >>>
>>>> >>>Thanks,
>>>> >>>Koushik
>>>> >>
>>>> >
>>>
>>
>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Kelven Yang <ke...@citrix.com>]

Nitin,

Sorry to reply late, have been busy working on a patch release for
customer. 

AsyncJob manager does provide a mechanism that you can do synchronized the
job execution against a object. You may check out
AsyncJobManager.syncAsyncJobExecution().

Kelven

On 3/12/13 11:27 AM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>syncAsyncJobExecutionThanks Alex.
>
>Kelven / Alex - I took a look into this and it seems that the framework
>has the capability to do this and this would solve the problem for scaling
>up vm and vm snapshots.
>But, we currently don't do synchronization for the vm object. Doing this
>means all the vm operations will be syncronized now. Are we fine with that
>?  
>
>Thanks,
>-Nitin
>
>On 09/03/13 10:36 PM, "Alex Huang" <Al...@citrix.com> wrote:
>
>>Nitin,
>>
>>The other approach to this is to utilize the syncing feature in the job
>>queue.  I've cced Kelven to see if he can give you more detail.  His code
>>is capable of syncing operations on a single object so you don't have to
>>add processing states.
>>
>>Given that all of your operations and vm snapshot operations must have
>>come in through the job queue, you might already have that ability to not
>>interfere with each other.
>>
>>VM States are different because there can be outside changes (through
>>other vm managers) that cause vm life cycle to behave differently.
>>
>>--Alex
>>
>>> -----Original Message-----
>>> From: Nitin Mehta
>>> Sent: Saturday, March 9, 2013 2:35 AM
>>> To: cloudstack-dev@incubator.apache.org; Prashant Kumar Mishra;
>>> Abhinandan Prateek; Alex Huang
>>> Cc: Chip Childers
>>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> 
>>> Hi Alex,
>>> I had one more question. Say the MS is shut down or restarted, when do
>>>we
>>> clear this attribute in this case ?
>>> 
>>> On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>>> 
>>> >Alex - Thanks very much for pointing out earlier this week that for
>>> >scaling up the vm we shouldn't change the vm lifecycle. I also read
>>> >http://markmail.org/message/6c6njactsklot62h
>>> >and understand that scaling up a vm is a vm operation and shouldn't be
>>> >mixed with vm lifecycle. So as you suggested in the thread that if I
>>> >need to prevent other Vm operations happening during this operation I
>>> >would need to introduce an attribute
>>> >
>>> >1. For this I would need to introduce a column in vm_instance table
>>> >which would be set during scale up operation.
>>> >2. To prevent other operations from happening this attribute needs to
>>> >be checked in all the other vm operations. There is no single common
>>> >piece of code where I can put the check so I have to explicitly check
>>> >for this attribute in all the operations code right ? I see that for
>>>"vm
>>> snapshot"
>>> >operation we have put this check in vm state transition method but
>>>this
>>> >method is called only for vm lifecycle changes. So when "vm snapshot"
>>> >happens the user might also scale up the vm. There might be a need for
>>> >them to be exclusive.
>>> >3. If I need to say lock capacity before the operation and modify it
>>> >after the operation is done (say during failure) how do I do it w/o
>>> >coupling the code changes or is it ok for now to do so ?
>>> >
>>> >
>>> >Thanks,
>>> >-Nitin
>>> >
>>> >On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
>>> >
>>> >>Hi Nitin,
>>> >>
>>> >>Please see below
>>> >>
>>> >>Hari
>>> >>
>>> >>-----Original Message-----
>>> >>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
>>> >>Sent: Tuesday, February 12, 2013 7:15 AM
>>> >>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
>>> >>Abhinandan Prateek
>>> >>Cc: Chip Childers
>>> >>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> >>
>>> >>Apologize for the delayed response. Was involved in other issues.
>>> >>Please find answers inline.
>>> >>
>>> >>>
>>> >>>-----Original Message-----
>>> >>>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
>>> >>>Sent: Thursday, January 24, 2013 12:26 PM
>>> >>>To: cloudstack-dev@incubator.apache.org
>>> >>>Cc: Nitin Mehta
>>> >>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> >>>
>>> >>>Hi Nitin,
>>> >>>I am planning to take the QA job for this feature. Have reviewed the
>>> >>>functional spec, gone through community discussion  and have the
>>> >>>following questions
>>> >>>
>>> >>>1-What is expected behavior of CS for Operating systems which do not
>>> >>>support dynamic scaling . ?
>>> >>
>>> >>Will throw a not supported exception
>>> >>
>>> >>Hari: How do we know which OS is supported or not? Is it going to be
>>> >>part of the "capabilities" of hypervisor? Or where will this be
>>> >>specified/configured?
>>> >>PS: I know we plan to implement this only on VMware for now, but when
>>> >>installed/shipped, how will CS know the supported Hypervisor/OS?
>>> >>
>>> >>>
>>> >>>2-How much resources can be scaled up, is it limited by availability
>>> >>>of resource on host .?
>>> >>>
>>> >>>[Koushik Das ]
>>> >>>"Having a range for CPU/RAM in compute offering is definitely
>>>another
>>> >>>way of doing it. But creating the higher limit would be tricky. I am
>>> >>>not sure if it is always known to users how much they want to scale
>>> >>>up to at the time of deploying VM. Moreover if the higher limit is
>>> >>>known then the VM can be deployed with that value itself. Also in
>>> >>>case of having a range in the offering the usage part needs to be
>>> >>>handled appropriately. Currently usage is purely based on the
>>> >>>offering and individual values are not stored".
>>> >>>[/Koushik Das]
>>> >>
>>> >>It is not limited by the resources on host but on the available
>>> >>capacity in any of the host within the cluster.
>>> >>
>>> >>Hari: I'm not sure I understand the question - how is this any
>>> >>different than requesting a new VM? Or upgrading from offering A to
>>> >>Offering B that exists today (although VM needs to be shutdown)?
>>> >>
>>> >>>
>>> >>>it seems  its totally depend on service offering , please correct me
>>> >>>if I am wrong.
>>> >>>
>>> >>>3-  Scheduled snapshot of volumes during the operation .
>>> >>>
>>> >>>[NITIN]
>>> >>>For vmware, the entire vm is locked by HV and this can be an issue.
>>>I
>>> >>>will leverage on current implementations for existing interactions
>>> >>>like scheduled snapshots events during live migration and will
>>> >>>replicate the same.
>>> >>>[/NITIN]
>>> >>>
>>> >>>Can you elaborate what is expected in case of VMware .
>>> >>
>>> >>What I mean is there is an existing functionality which is
>>>implemented
>>> >>the same way. I will just do it the same way.
>>> >>
>>> >>>
>>> >>>4 - what is expected behavior in case of  powers off the vm during
>>> >>>the operation .? is it different for different hypervisors.?
>>> >>
>>> >>There is nothing much to do for powered off vms because we will just
>>> >>update the DB. When the vm is started it will pick up these values
>>> >>from the DB.
>>> >>This functionality already exists.
>>> >>
>>> >>>
>>> >>>5- what is expected in case of migration fails( In FS no description
>>> >>>about this),
>>> >>>       -CS will  retry to migrate it again if yes how many time ?
>>> >>>      - will it mark as a failure and can't  scale up(even resources
>>> >>>are available in cluster) ?
>>> >>
>>> >>We will retry N (configurable) times and if unsuccessful we will
>>>throw
>>> >>an exception to the user.
>>> >>
>>> >>Hari: Can you please elaborate why a migration might fail? And, is
>>>the
>>> >>"N" configurable times a new global?
>>> >>
>>> >>>
>>> >>>6- Apart from  "scaleVirtualMachine"    any other APIs are getting
>>> >>>changed ?
>>> >>
>>> >>No
>>> >>
>>> >>>
>>> >>>7-Scale down is allowed ? (still open issue in FS)
>>> >>
>>> >>No for time being.
>>> >>
>>> >>>
>>> >>>8-Are we going to introduce custom compute offering (still open
>>>issue
>>> >>>in
>>> >>>FS) ?
>>> >>
>>> >>No for now
>>> >>
>>> >>>
>>> >>>9- what are the guide line for upgrade  ?
>>> >>
>>> >>There is nothing for upgrade because we do not introduce new values
>>>in
>>> >>DB.
>>> >>
>>> >>>
>>> >>>10-Any DB changes ?
>>> >>
>>> >>See #9 above.
>>> >>
>>> >>>
>>> >>>11- which Usage events are getting introduced for billing .?
>>> >>
>>> >>Will update the FS.
>>> >>
>>> >>>
>>> >>>12-hypervisor support ,is it only for VMware (as per FS)  or its
>>> >>>getting extended for XS/KVM also ?
>>> >>
>>> >>There are subtasks opened for XS and KVM. I am doing it only for
>>>Vmware.
>>> >>
>>> >>>
>>> >>>Thanks
>>> >>>Prashant Kumar Mishra
>>> >>>
>>> >>>
>>> >>>-----Original Message-----
>>> >>>From: Koushik Das [mailto:koushik.das@citrix.com]
>>> >>>Sent: Saturday, December 15, 2012 11:14 PM
>>> >>>To: cloudstack-dev@incubator.apache.org
>>> >>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>> >>>
>>> >>>Currently CS supports changing CPU and RAM for stopped VM. This is
>>> >>>achieved by changing compute offering of the VM (with new CPU and
>>> RAM
>>> >>>values) and then starting it. I am planning to extend the same for
>>> >>>running VM as well. Initially planning to do it for Vmware where CPU
>>> >>>and RAM can be dynamically increased. Support of other HVs can also
>>> >>>be added if they support increasing CPU/RAM.
>>> >>>
>>> >>>Assuming that in the updated compute offering only CPU and RAM has
>>> >>>changed, the deployment planner can either select the same host in
>>> >>>which case the values are dynamically scaled up OR a different one
>>>in
>>> >>>which case the operation fails. In future if there is support for
>>> >>>live migration (provided HV supports it) then another option in the
>>> >>>latter case could be to migrate the VM first and then scale it up.
>>> >>>
>>> >>>I will start working on the FS and share it out sometime next week.
>>> >>>
>>> >>>Comments/suggestions?
>>> >>>
>>> >>>Thanks,
>>> >>>Koushik
>>> >>
>>> >
>>
>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Nitin Mehta <Ni...@citrix.com>]

Thanks Alex.

Kelven / Alex - I took a look into this and it seems that the framework
has the capability to do this and this would solve the problem for scaling
up vm and vm snapshots.
But, we currently don't do synchronization for the vm object. Doing this
means all the vm operations will be syncronized now. Are we fine with that
?  

Thanks,
-Nitin

On 09/03/13 10:36 PM, "Alex Huang" <Al...@citrix.com> wrote:

>Nitin,
>
>The other approach to this is to utilize the syncing feature in the job
>queue.  I've cced Kelven to see if he can give you more detail.  His code
>is capable of syncing operations on a single object so you don't have to
>add processing states.
>
>Given that all of your operations and vm snapshot operations must have
>come in through the job queue, you might already have that ability to not
>interfere with each other.
>
>VM States are different because there can be outside changes (through
>other vm managers) that cause vm life cycle to behave differently.
>
>--Alex
>
>> -----Original Message-----
>> From: Nitin Mehta
>> Sent: Saturday, March 9, 2013 2:35 AM
>> To: cloudstack-dev@incubator.apache.org; Prashant Kumar Mishra;
>> Abhinandan Prateek; Alex Huang
>> Cc: Chip Childers
>> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>> 
>> Hi Alex,
>> I had one more question. Say the MS is shut down or restarted, when do
>>we
>> clear this attribute in this case ?
>> 
>> On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
>> 
>> >Alex - Thanks very much for pointing out earlier this week that for
>> >scaling up the vm we shouldn't change the vm lifecycle. I also read
>> >http://markmail.org/message/6c6njactsklot62h
>> >and understand that scaling up a vm is a vm operation and shouldn't be
>> >mixed with vm lifecycle. So as you suggested in the thread that if I
>> >need to prevent other Vm operations happening during this operation I
>> >would need to introduce an attribute
>> >
>> >1. For this I would need to introduce a column in vm_instance table
>> >which would be set during scale up operation.
>> >2. To prevent other operations from happening this attribute needs to
>> >be checked in all the other vm operations. There is no single common
>> >piece of code where I can put the check so I have to explicitly check
>> >for this attribute in all the operations code right ? I see that for
>>"vm
>> snapshot"
>> >operation we have put this check in vm state transition method but this
>> >method is called only for vm lifecycle changes. So when "vm snapshot"
>> >happens the user might also scale up the vm. There might be a need for
>> >them to be exclusive.
>> >3. If I need to say lock capacity before the operation and modify it
>> >after the operation is done (say during failure) how do I do it w/o
>> >coupling the code changes or is it ok for now to do so ?
>> >
>> >
>> >Thanks,
>> >-Nitin
>> >
>> >On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
>> >
>> >>Hi Nitin,
>> >>
>> >>Please see below
>> >>
>> >>Hari
>> >>
>> >>-----Original Message-----
>> >>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
>> >>Sent: Tuesday, February 12, 2013 7:15 AM
>> >>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
>> >>Abhinandan Prateek
>> >>Cc: Chip Childers
>> >>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>> >>
>> >>Apologize for the delayed response. Was involved in other issues.
>> >>Please find answers inline.
>> >>
>> >>>
>> >>>-----Original Message-----
>> >>>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
>> >>>Sent: Thursday, January 24, 2013 12:26 PM
>> >>>To: cloudstack-dev@incubator.apache.org
>> >>>Cc: Nitin Mehta
>> >>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>> >>>
>> >>>Hi Nitin,
>> >>>I am planning to take the QA job for this feature. Have reviewed the
>> >>>functional spec, gone through community discussion  and have the
>> >>>following questions
>> >>>
>> >>>1-What is expected behavior of CS for Operating systems which do not
>> >>>support dynamic scaling . ?
>> >>
>> >>Will throw a not supported exception
>> >>
>> >>Hari: How do we know which OS is supported or not? Is it going to be
>> >>part of the "capabilities" of hypervisor? Or where will this be
>> >>specified/configured?
>> >>PS: I know we plan to implement this only on VMware for now, but when
>> >>installed/shipped, how will CS know the supported Hypervisor/OS?
>> >>
>> >>>
>> >>>2-How much resources can be scaled up, is it limited by availability
>> >>>of resource on host .?
>> >>>
>> >>>[Koushik Das ]
>> >>>"Having a range for CPU/RAM in compute offering is definitely another
>> >>>way of doing it. But creating the higher limit would be tricky. I am
>> >>>not sure if it is always known to users how much they want to scale
>> >>>up to at the time of deploying VM. Moreover if the higher limit is
>> >>>known then the VM can be deployed with that value itself. Also in
>> >>>case of having a range in the offering the usage part needs to be
>> >>>handled appropriately. Currently usage is purely based on the
>> >>>offering and individual values are not stored".
>> >>>[/Koushik Das]
>> >>
>> >>It is not limited by the resources on host but on the available
>> >>capacity in any of the host within the cluster.
>> >>
>> >>Hari: I'm not sure I understand the question - how is this any
>> >>different than requesting a new VM? Or upgrading from offering A to
>> >>Offering B that exists today (although VM needs to be shutdown)?
>> >>
>> >>>
>> >>>it seems  its totally depend on service offering , please correct me
>> >>>if I am wrong.
>> >>>
>> >>>3-  Scheduled snapshot of volumes during the operation .
>> >>>
>> >>>[NITIN]
>> >>>For vmware, the entire vm is locked by HV and this can be an issue. I
>> >>>will leverage on current implementations for existing interactions
>> >>>like scheduled snapshots events during live migration and will
>> >>>replicate the same.
>> >>>[/NITIN]
>> >>>
>> >>>Can you elaborate what is expected in case of VMware .
>> >>
>> >>What I mean is there is an existing functionality which is implemented
>> >>the same way. I will just do it the same way.
>> >>
>> >>>
>> >>>4 - what is expected behavior in case of  powers off the vm during
>> >>>the operation .? is it different for different hypervisors.?
>> >>
>> >>There is nothing much to do for powered off vms because we will just
>> >>update the DB. When the vm is started it will pick up these values
>> >>from the DB.
>> >>This functionality already exists.
>> >>
>> >>>
>> >>>5- what is expected in case of migration fails( In FS no description
>> >>>about this),
>> >>>       -CS will  retry to migrate it again if yes how many time ?
>> >>>      - will it mark as a failure and can't  scale up(even resources
>> >>>are available in cluster) ?
>> >>
>> >>We will retry N (configurable) times and if unsuccessful we will throw
>> >>an exception to the user.
>> >>
>> >>Hari: Can you please elaborate why a migration might fail? And, is the
>> >>"N" configurable times a new global?
>> >>
>> >>>
>> >>>6- Apart from  "scaleVirtualMachine"    any other APIs are getting
>> >>>changed ?
>> >>
>> >>No
>> >>
>> >>>
>> >>>7-Scale down is allowed ? (still open issue in FS)
>> >>
>> >>No for time being.
>> >>
>> >>>
>> >>>8-Are we going to introduce custom compute offering (still open issue
>> >>>in
>> >>>FS) ?
>> >>
>> >>No for now
>> >>
>> >>>
>> >>>9- what are the guide line for upgrade  ?
>> >>
>> >>There is nothing for upgrade because we do not introduce new values in
>> >>DB.
>> >>
>> >>>
>> >>>10-Any DB changes ?
>> >>
>> >>See #9 above.
>> >>
>> >>>
>> >>>11- which Usage events are getting introduced for billing .?
>> >>
>> >>Will update the FS.
>> >>
>> >>>
>> >>>12-hypervisor support ,is it only for VMware (as per FS)  or its
>> >>>getting extended for XS/KVM also ?
>> >>
>> >>There are subtasks opened for XS and KVM. I am doing it only for
>>Vmware.
>> >>
>> >>>
>> >>>Thanks
>> >>>Prashant Kumar Mishra
>> >>>
>> >>>
>> >>>-----Original Message-----
>> >>>From: Koushik Das [mailto:koushik.das@citrix.com]
>> >>>Sent: Saturday, December 15, 2012 11:14 PM
>> >>>To: cloudstack-dev@incubator.apache.org
>> >>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>> >>>
>> >>>Currently CS supports changing CPU and RAM for stopped VM. This is
>> >>>achieved by changing compute offering of the VM (with new CPU and
>> RAM
>> >>>values) and then starting it. I am planning to extend the same for
>> >>>running VM as well. Initially planning to do it for Vmware where CPU
>> >>>and RAM can be dynamically increased. Support of other HVs can also
>> >>>be added if they support increasing CPU/RAM.
>> >>>
>> >>>Assuming that in the updated compute offering only CPU and RAM has
>> >>>changed, the deployment planner can either select the same host in
>> >>>which case the values are dynamically scaled up OR a different one in
>> >>>which case the operation fails. In future if there is support for
>> >>>live migration (provided HV supports it) then another option in the
>> >>>latter case could be to migrate the VM first and then scale it up.
>> >>>
>> >>>I will start working on the FS and share it out sometime next week.
>> >>>
>> >>>Comments/suggestions?
>> >>>
>> >>>Thanks,
>> >>>Koushik
>> >>
>> >
>

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Alex Huang <Al...@citrix.com>]

Nitin,

The other approach to this is to utilize the syncing feature in the job queue.  I've cced Kelven to see if he can give you more detail.  His code is capable of syncing operations on a single object so you don't have to add processing states.

Given that all of your operations and vm snapshot operations must have come in through the job queue, you might already have that ability to not interfere with each other.  

VM States are different because there can be outside changes (through other vm managers) that cause vm life cycle to behave differently.

--Alex

> -----Original Message-----
> From: Nitin Mehta
> Sent: Saturday, March 9, 2013 2:35 AM
> To: cloudstack-dev@incubator.apache.org; Prashant Kumar Mishra;
> Abhinandan Prateek; Alex Huang
> Cc: Chip Childers
> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> 
> Hi Alex,
> I had one more question. Say the MS is shut down or restarted, when do we
> clear this attribute in this case ?
> 
> On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:
> 
> >Alex - Thanks very much for pointing out earlier this week that for
> >scaling up the vm we shouldn't change the vm lifecycle. I also read
> >http://markmail.org/message/6c6njactsklot62h
> >and understand that scaling up a vm is a vm operation and shouldn't be
> >mixed with vm lifecycle. So as you suggested in the thread that if I
> >need to prevent other Vm operations happening during this operation I
> >would need to introduce an attribute
> >
> >1. For this I would need to introduce a column in vm_instance table
> >which would be set during scale up operation.
> >2. To prevent other operations from happening this attribute needs to
> >be checked in all the other vm operations. There is no single common
> >piece of code where I can put the check so I have to explicitly check
> >for this attribute in all the operations code right ? I see that for "vm
> snapshot"
> >operation we have put this check in vm state transition method but this
> >method is called only for vm lifecycle changes. So when "vm snapshot"
> >happens the user might also scale up the vm. There might be a need for
> >them to be exclusive.
> >3. If I need to say lock capacity before the operation and modify it
> >after the operation is done (say during failure) how do I do it w/o
> >coupling the code changes or is it ok for now to do so ?
> >
> >
> >Thanks,
> >-Nitin
> >
> >On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
> >
> >>Hi Nitin,
> >>
> >>Please see below
> >>
> >>Hari
> >>
> >>-----Original Message-----
> >>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
> >>Sent: Tuesday, February 12, 2013 7:15 AM
> >>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
> >>Abhinandan Prateek
> >>Cc: Chip Childers
> >>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>
> >>Apologize for the delayed response. Was involved in other issues.
> >>Please find answers inline.
> >>
> >>>
> >>>-----Original Message-----
> >>>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
> >>>Sent: Thursday, January 24, 2013 12:26 PM
> >>>To: cloudstack-dev@incubator.apache.org
> >>>Cc: Nitin Mehta
> >>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>
> >>>Hi Nitin,
> >>>I am planning to take the QA job for this feature. Have reviewed the
> >>>functional spec, gone through community discussion  and have the
> >>>following questions
> >>>
> >>>1-What is expected behavior of CS for Operating systems which do not
> >>>support dynamic scaling . ?
> >>
> >>Will throw a not supported exception
> >>
> >>Hari: How do we know which OS is supported or not? Is it going to be
> >>part of the "capabilities" of hypervisor? Or where will this be
> >>specified/configured?
> >>PS: I know we plan to implement this only on VMware for now, but when
> >>installed/shipped, how will CS know the supported Hypervisor/OS?
> >>
> >>>
> >>>2-How much resources can be scaled up, is it limited by availability
> >>>of resource on host .?
> >>>
> >>>[Koushik Das ]
> >>>"Having a range for CPU/RAM in compute offering is definitely another
> >>>way of doing it. But creating the higher limit would be tricky. I am
> >>>not sure if it is always known to users how much they want to scale
> >>>up to at the time of deploying VM. Moreover if the higher limit is
> >>>known then the VM can be deployed with that value itself. Also in
> >>>case of having a range in the offering the usage part needs to be
> >>>handled appropriately. Currently usage is purely based on the
> >>>offering and individual values are not stored".
> >>>[/Koushik Das]
> >>
> >>It is not limited by the resources on host but on the available
> >>capacity in any of the host within the cluster.
> >>
> >>Hari: I'm not sure I understand the question - how is this any
> >>different than requesting a new VM? Or upgrading from offering A to
> >>Offering B that exists today (although VM needs to be shutdown)?
> >>
> >>>
> >>>it seems  its totally depend on service offering , please correct me
> >>>if I am wrong.
> >>>
> >>>3-  Scheduled snapshot of volumes during the operation .
> >>>
> >>>[NITIN]
> >>>For vmware, the entire vm is locked by HV and this can be an issue. I
> >>>will leverage on current implementations for existing interactions
> >>>like scheduled snapshots events during live migration and will
> >>>replicate the same.
> >>>[/NITIN]
> >>>
> >>>Can you elaborate what is expected in case of VMware .
> >>
> >>What I mean is there is an existing functionality which is implemented
> >>the same way. I will just do it the same way.
> >>
> >>>
> >>>4 - what is expected behavior in case of  powers off the vm during
> >>>the operation .? is it different for different hypervisors.?
> >>
> >>There is nothing much to do for powered off vms because we will just
> >>update the DB. When the vm is started it will pick up these values
> >>from the DB.
> >>This functionality already exists.
> >>
> >>>
> >>>5- what is expected in case of migration fails( In FS no description
> >>>about this),
> >>>       -CS will  retry to migrate it again if yes how many time ?
> >>>      - will it mark as a failure and can't  scale up(even resources
> >>>are available in cluster) ?
> >>
> >>We will retry N (configurable) times and if unsuccessful we will throw
> >>an exception to the user.
> >>
> >>Hari: Can you please elaborate why a migration might fail? And, is the
> >>"N" configurable times a new global?
> >>
> >>>
> >>>6- Apart from  "scaleVirtualMachine"    any other APIs are getting
> >>>changed ?
> >>
> >>No
> >>
> >>>
> >>>7-Scale down is allowed ? (still open issue in FS)
> >>
> >>No for time being.
> >>
> >>>
> >>>8-Are we going to introduce custom compute offering (still open issue
> >>>in
> >>>FS) ?
> >>
> >>No for now
> >>
> >>>
> >>>9- what are the guide line for upgrade  ?
> >>
> >>There is nothing for upgrade because we do not introduce new values in
> >>DB.
> >>
> >>>
> >>>10-Any DB changes ?
> >>
> >>See #9 above.
> >>
> >>>
> >>>11- which Usage events are getting introduced for billing .?
> >>
> >>Will update the FS.
> >>
> >>>
> >>>12-hypervisor support ,is it only for VMware (as per FS)  or its
> >>>getting extended for XS/KVM also ?
> >>
> >>There are subtasks opened for XS and KVM. I am doing it only for Vmware.
> >>
> >>>
> >>>Thanks
> >>>Prashant Kumar Mishra
> >>>
> >>>
> >>>-----Original Message-----
> >>>From: Koushik Das [mailto:koushik.das@citrix.com]
> >>>Sent: Saturday, December 15, 2012 11:14 PM
> >>>To: cloudstack-dev@incubator.apache.org
> >>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> >>>
> >>>Currently CS supports changing CPU and RAM for stopped VM. This is
> >>>achieved by changing compute offering of the VM (with new CPU and
> RAM
> >>>values) and then starting it. I am planning to extend the same for
> >>>running VM as well. Initially planning to do it for Vmware where CPU
> >>>and RAM can be dynamically increased. Support of other HVs can also
> >>>be added if they support increasing CPU/RAM.
> >>>
> >>>Assuming that in the updated compute offering only CPU and RAM has
> >>>changed, the deployment planner can either select the same host in
> >>>which case the values are dynamically scaled up OR a different one in
> >>>which case the operation fails. In future if there is support for
> >>>live migration (provided HV supports it) then another option in the
> >>>latter case could be to migrate the VM first and then scale it up.
> >>>
> >>>I will start working on the FS and share it out sometime next week.
> >>>
> >>>Comments/suggestions?
> >>>
> >>>Thanks,
> >>>Koushik
> >>
> >

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Nitin Mehta <Ni...@citrix.com>]

Hi Alex,
I had one more question. Say the MS is shut down or restarted, when do we
clear this attribute in this case ?

On 08/03/13 6:01 PM, "Nitin Mehta" <Ni...@citrix.com> wrote:

>Alex - Thanks very much for pointing out earlier this week that for
>scaling up the vm we shouldn't change the vm lifecycle. I also read
>http://markmail.org/message/6c6njactsklot62h
>and understand that scaling up a vm is a vm operation and shouldn't be
>mixed with vm lifecycle. So as you suggested in the thread that if I need
>to prevent other 
>Vm operations happening during this operation I would need to introduce an
>attribute
>
>1. For this I would need to introduce a column in vm_instance table which
>would be set during scale up operation.
>2. To prevent other operations from happening this attribute needs to be
>checked in all the other vm operations. There is no single common piece of
>code where I can put the check so I have to explicitly check for this
>attribute in all the operations code right ? I see that for "vm snapshot"
>operation we have put this check in vm state transition method but this
>method is called only for vm lifecycle changes. So when "vm snapshot"
>happens the user might also scale up the vm. There might be a need for
>them to be exclusive.
>3. If I need to say lock capacity before the operation and modify it after
>the operation is done (say during failure) how do I do it w/o coupling the
>code changes or is it ok for now to do so ?
>
>
>Thanks,
>-Nitin
>
>On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:
>
>>Hi Nitin,
>>
>>Please see below
>>
>>Hari
>>
>>-----Original Message-----
>>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
>>Sent: Tuesday, February 12, 2013 7:15 AM
>>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
>>Abhinandan Prateek
>>Cc: Chip Childers
>>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>>
>>Apologize for the delayed response. Was involved in other issues.
>>Please find answers inline.
>>
>>>
>>>-----Original Message-----
>>>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
>>>Sent: Thursday, January 24, 2013 12:26 PM
>>>To: cloudstack-dev@incubator.apache.org
>>>Cc: Nitin Mehta
>>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>
>>>Hi Nitin,
>>>I am planning to take the QA job for this feature. Have reviewed the
>>>functional spec, gone through community discussion  and have the
>>>following questions
>>>
>>>1-What is expected behavior of CS for Operating systems which do not
>>>support dynamic scaling . ?
>>
>>Will throw a not supported exception
>>
>>Hari: How do we know which OS is supported or not? Is it going to be part
>>of the "capabilities" of hypervisor? Or where will this be
>>specified/configured?
>>PS: I know we plan to implement this only on VMware for now, but when
>>installed/shipped, how will CS know the supported Hypervisor/OS?
>>
>>> 
>>>2-How much resources can be scaled up, is it limited by availability of
>>>resource on host .?
>>> 
>>>[Koushik Das ]
>>>"Having a range for CPU/RAM in compute offering is definitely another
>>>way of doing it. But creating the higher limit would be tricky. I am
>>>not sure if it is always known to users how much they want to scale up
>>>to at the time of deploying VM. Moreover if the higher limit is known
>>>then the VM can be deployed with that value itself. Also in case of
>>>having a range in the offering the usage part needs to be handled
>>>appropriately. Currently usage is purely based on the offering and
>>>individual values are not stored".
>>>[/Koushik Das]
>>
>>It is not limited by the resources on host but on the available capacity
>>in any of the host within the cluster.
>>
>>Hari: I'm not sure I understand the question - how is this any different
>>than requesting a new VM? Or upgrading from offering A to Offering B that
>>exists today (although VM needs to be shutdown)?
>>
>>>
>>>it seems  its totally depend on service offering , please correct me if
>>>I am wrong.
>>>
>>>3-  Scheduled snapshot of volumes during the operation .
>>>
>>>[NITIN]
>>>For vmware, the entire vm is locked by HV and this can be an issue. I
>>>will leverage on current implementations for existing interactions like
>>>scheduled snapshots events during live migration and will replicate the
>>>same.
>>>[/NITIN]
>>>
>>>Can you elaborate what is expected in case of VMware .
>>
>>What I mean is there is an existing functionality which is implemented
>>the same way. I will just do it the same way.
>>
>>>
>>>4 - what is expected behavior in case of  powers off the vm during the
>>>operation .? is it different for different hypervisors.?
>>
>>There is nothing much to do for powered off vms because we will just
>>update the DB. When the vm is started it will pick up these values from
>>the DB.
>>This functionality already exists.
>>
>>>
>>>5- what is expected in case of migration fails( In FS no description
>>>about this),
>>>       -CS will  retry to migrate it again if yes how many time ?
>>>      - will it mark as a failure and can't  scale up(even resources
>>>are available in cluster) ?
>>
>>We will retry N (configurable) times and if unsuccessful we will throw an
>>exception to the user.
>>
>>Hari: Can you please elaborate why a migration might fail? And, is the
>>"N" configurable times a new global?
>>
>>>
>>>6- Apart from  "scaleVirtualMachine"    any other APIs are getting
>>>changed ?
>>
>>No 
>>
>>>
>>>7-Scale down is allowed ? (still open issue in FS)
>>
>>No for time being.
>>
>>>
>>>8-Are we going to introduce custom compute offering (still open issue
>>>in
>>>FS) ?
>>
>>No for now 
>>
>>>
>>>9- what are the guide line for upgrade  ?
>>
>>There is nothing for upgrade because we do not introduce new values in
>>DB.
>>
>>>
>>>10-Any DB changes ?
>>
>>See #9 above.
>>
>>>
>>>11- which Usage events are getting introduced for billing .?
>>
>>Will update the FS.
>>
>>>
>>>12-hypervisor support ,is it only for VMware (as per FS)  or its
>>>getting extended for XS/KVM also ?
>>
>>There are subtasks opened for XS and KVM. I am doing it only for Vmware.
>>
>>>
>>>Thanks
>>>Prashant Kumar Mishra
>>>
>>>
>>>-----Original Message-----
>>>From: Koushik Das [mailto:koushik.das@citrix.com]
>>>Sent: Saturday, December 15, 2012 11:14 PM
>>>To: cloudstack-dev@incubator.apache.org
>>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>>
>>>Currently CS supports changing CPU and RAM for stopped VM. This is
>>>achieved by changing compute offering of the VM (with new CPU and RAM
>>>values) and then starting it. I am planning to extend the same for
>>>running VM as well. Initially planning to do it for Vmware where CPU
>>>and RAM can be dynamically increased. Support of other HVs can also be
>>>added if they support increasing CPU/RAM.
>>>
>>>Assuming that in the updated compute offering only CPU and RAM has
>>>changed, the deployment planner can either select the same host in
>>>which case the values are dynamically scaled up OR a different one in
>>>which case the operation fails. In future if there is support for live
>>>migration (provided HV supports it) then another option in the latter
>>>case could be to migrate the VM first and then scale it up.
>>>
>>>I will start working on the FS and share it out sometime next week.
>>>
>>>Comments/suggestions?
>>>
>>>Thanks,
>>>Koushik
>>
>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Nitin Mehta <Ni...@citrix.com>]

Alex - Thanks very much for pointing out earlier this week that for
scaling up the vm we shouldn't change the vm lifecycle. I also read
http://markmail.org/message/6c6njactsklot62h
and understand that scaling up a vm is a vm operation and shouldn't be
mixed with vm lifecycle. So as you suggested in the thread that if I need
to prevent other 
Vm operations happening during this operation I would need to introduce an
attribute

1. For this I would need to introduce a column in vm_instance table which
would be set during scale up operation.
2. To prevent other operations from happening this attribute needs to be
checked in all the other vm operations. There is no single common piece of
code where I can put the check so I have to explicitly check for this
attribute in all the operations code right ? I see that for "vm snapshot"
operation we have put this check in vm state transition method but this
method is called only for vm lifecycle changes. So when "vm snapshot"
happens the user might also scale up the vm. There might be a need for
them to be exclusive.
3. If I need to say lock capacity before the operation and modify it after
the operation is done (say during failure) how do I do it w/o coupling the
code changes or is it ok for now to do so ?


Thanks,
-Nitin

On 15/02/13 5:42 AM, "Hari Kannan" <ha...@citrix.com> wrote:

>Hi Nitin,
>
>Please see below
>
>Hari
>
>-----Original Message-----
>From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com]
>Sent: Tuesday, February 12, 2013 7:15 AM
>To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org;
>Abhinandan Prateek
>Cc: Chip Childers
>Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
>
>Apologize for the delayed response. Was involved in other issues.
>Please find answers inline.
>
>>
>>-----Original Message-----
>>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
>>Sent: Thursday, January 24, 2013 12:26 PM
>>To: cloudstack-dev@incubator.apache.org
>>Cc: Nitin Mehta
>>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>>
>>Hi Nitin,
>>I am planning to take the QA job for this feature. Have reviewed the
>>functional spec, gone through community discussion  and have the
>>following questions
>>
>>1-What is expected behavior of CS for Operating systems which do not
>>support dynamic scaling . ?
>
>Will throw a not supported exception
>
>Hari: How do we know which OS is supported or not? Is it going to be part
>of the "capabilities" of hypervisor? Or where will this be
>specified/configured?
>PS: I know we plan to implement this only on VMware for now, but when
>installed/shipped, how will CS know the supported Hypervisor/OS?
>
>> 
>>2-How much resources can be scaled up, is it limited by availability of
>>resource on host .?
>> 
>>[Koushik Das ]
>>"Having a range for CPU/RAM in compute offering is definitely another
>>way of doing it. But creating the higher limit would be tricky. I am
>>not sure if it is always known to users how much they want to scale up
>>to at the time of deploying VM. Moreover if the higher limit is known
>>then the VM can be deployed with that value itself. Also in case of
>>having a range in the offering the usage part needs to be handled
>>appropriately. Currently usage is purely based on the offering and
>>individual values are not stored".
>>[/Koushik Das]
>
>It is not limited by the resources on host but on the available capacity
>in any of the host within the cluster.
>
>Hari: I'm not sure I understand the question - how is this any different
>than requesting a new VM? Or upgrading from offering A to Offering B that
>exists today (although VM needs to be shutdown)?
>
>>
>>it seems  its totally depend on service offering , please correct me if
>>I am wrong.
>>
>>3-  Scheduled snapshot of volumes during the operation .
>>
>>[NITIN]
>>For vmware, the entire vm is locked by HV and this can be an issue. I
>>will leverage on current implementations for existing interactions like
>>scheduled snapshots events during live migration and will replicate the
>>same.
>>[/NITIN]
>>
>>Can you elaborate what is expected in case of VMware .
>
>What I mean is there is an existing functionality which is implemented
>the same way. I will just do it the same way.
>
>>
>>4 - what is expected behavior in case of  powers off the vm during the
>>operation .? is it different for different hypervisors.?
>
>There is nothing much to do for powered off vms because we will just
>update the DB. When the vm is started it will pick up these values from
>the DB.
>This functionality already exists.
>
>>
>>5- what is expected in case of migration fails( In FS no description
>>about this),
>>       -CS will  retry to migrate it again if yes how many time ?
>>      - will it mark as a failure and can't  scale up(even resources
>>are available in cluster) ?
>
>We will retry N (configurable) times and if unsuccessful we will throw an
>exception to the user.
>
>Hari: Can you please elaborate why a migration might fail? And, is the
>"N" configurable times a new global?
>
>>
>>6- Apart from  "scaleVirtualMachine"    any other APIs are getting
>>changed ?
>
>No 
>
>>
>>7-Scale down is allowed ? (still open issue in FS)
>
>No for time being.
>
>>
>>8-Are we going to introduce custom compute offering (still open issue
>>in
>>FS) ?
>
>No for now 
>
>>
>>9- what are the guide line for upgrade  ?
>
>There is nothing for upgrade because we do not introduce new values in DB.
>
>>
>>10-Any DB changes ?
>
>See #9 above.
>
>>
>>11- which Usage events are getting introduced for billing .?
>
>Will update the FS.
>
>>
>>12-hypervisor support ,is it only for VMware (as per FS)  or its
>>getting extended for XS/KVM also ?
>
>There are subtasks opened for XS and KVM. I am doing it only for Vmware.
>
>>
>>Thanks
>>Prashant Kumar Mishra
>>
>>
>>-----Original Message-----
>>From: Koushik Das [mailto:koushik.das@citrix.com]
>>Sent: Saturday, December 15, 2012 11:14 PM
>>To: cloudstack-dev@incubator.apache.org
>>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>>
>>Currently CS supports changing CPU and RAM for stopped VM. This is
>>achieved by changing compute offering of the VM (with new CPU and RAM
>>values) and then starting it. I am planning to extend the same for
>>running VM as well. Initially planning to do it for Vmware where CPU
>>and RAM can be dynamically increased. Support of other HVs can also be
>>added if they support increasing CPU/RAM.
>>
>>Assuming that in the updated compute offering only CPU and RAM has
>>changed, the deployment planner can either select the same host in
>>which case the values are dynamically scaled up OR a different one in
>>which case the operation fails. In future if there is support for live
>>migration (provided HV supports it) then another option in the latter
>>case could be to migrate the VM first and then scale it up.
>>
>>I will start working on the FS and share it out sometime next week.
>>
>>Comments/suggestions?
>>
>>Thanks,
>>Koushik
>

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Hari Kannan <ha...@citrix.com>]

Hi Nitin,

Please see below

Hari

-----Original Message-----
From: Nitin Mehta [mailto:Nitin.Mehta@citrix.com] 
Sent: Tuesday, February 12, 2013 7:15 AM
To: Prashant Kumar Mishra; cloudstack-dev@incubator.apache.org; Abhinandan Prateek
Cc: Chip Childers
Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs

Apologize for the delayed response. Was involved in other issues.
Please find answers inline.

>
>-----Original Message-----
>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
>Sent: Thursday, January 24, 2013 12:26 PM
>To: cloudstack-dev@incubator.apache.org
>Cc: Nitin Mehta
>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>
>Hi Nitin,
>I am planning to take the QA job for this feature. Have reviewed the 
>functional spec, gone through community discussion  and have the 
>following questions
>
>1-What is expected behavior of CS for Operating systems which do not 
>support dynamic scaling . ?

Will throw a not supported exception

Hari: How do we know which OS is supported or not? Is it going to be part of the "capabilities" of hypervisor? Or where will this be specified/configured?
PS: I know we plan to implement this only on VMware for now, but when installed/shipped, how will CS know the supported Hypervisor/OS?

> 
>2-How much resources can be scaled up, is it limited by availability of 
>resource on host .?
> 
>[Koushik Das ]
>"Having a range for CPU/RAM in compute offering is definitely another 
>way of doing it. But creating the higher limit would be tricky. I am 
>not sure if it is always known to users how much they want to scale up 
>to at the time of deploying VM. Moreover if the higher limit is known 
>then the VM can be deployed with that value itself. Also in case of 
>having a range in the offering the usage part needs to be handled 
>appropriately. Currently usage is purely based on the offering and 
>individual values are not stored".
>[/Koushik Das]

It is not limited by the resources on host but on the available capacity in any of the host within the cluster.

Hari: I'm not sure I understand the question - how is this any different than requesting a new VM? Or upgrading from offering A to Offering B that exists today (although VM needs to be shutdown)?

>
>it seems  its totally depend on service offering , please correct me if 
>I am wrong.
>
>3-  Scheduled snapshot of volumes during the operation .
>
>[NITIN]
>For vmware, the entire vm is locked by HV and this can be an issue. I 
>will leverage on current implementations for existing interactions like 
>scheduled snapshots events during live migration and will replicate the 
>same.
>[/NITIN]
>
>Can you elaborate what is expected in case of VMware .

What I mean is there is an existing functionality which is implemented the same way. I will just do it the same way.

>
>4 - what is expected behavior in case of  powers off the vm during the 
>operation .? is it different for different hypervisors.?

There is nothing much to do for powered off vms because we will just update the DB. When the vm is started it will pick up these values from the DB.
This functionality already exists.

>
>5- what is expected in case of migration fails( In FS no description 
>about this),
>       -CS will  retry to migrate it again if yes how many time ?
>      - will it mark as a failure and can't  scale up(even resources 
>are available in cluster) ?

We will retry N (configurable) times and if unsuccessful we will throw an exception to the user.

Hari: Can you please elaborate why a migration might fail? And, is the "N" configurable times a new global?

>
>6- Apart from  "scaleVirtualMachine"    any other APIs are getting
>changed ?

No 

>
>7-Scale down is allowed ? (still open issue in FS)

No for time being. 

>
>8-Are we going to introduce custom compute offering (still open issue 
>in
>FS) ?

No for now 

>
>9- what are the guide line for upgrade  ?

There is nothing for upgrade because we do not introduce new values in DB.

>
>10-Any DB changes ?

See #9 above.

>
>11- which Usage events are getting introduced for billing .?

Will update the FS.

>
>12-hypervisor support ,is it only for VMware (as per FS)  or its 
>getting extended for XS/KVM also ?

There are subtasks opened for XS and KVM. I am doing it only for Vmware.

>
>Thanks
>Prashant Kumar Mishra
>
>
>-----Original Message-----
>From: Koushik Das [mailto:koushik.das@citrix.com]
>Sent: Saturday, December 15, 2012 11:14 PM
>To: cloudstack-dev@incubator.apache.org
>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>
>Currently CS supports changing CPU and RAM for stopped VM. This is 
>achieved by changing compute offering of the VM (with new CPU and RAM
>values) and then starting it. I am planning to extend the same for 
>running VM as well. Initially planning to do it for Vmware where CPU 
>and RAM can be dynamically increased. Support of other HVs can also be 
>added if they support increasing CPU/RAM.
>
>Assuming that in the updated compute offering only CPU and RAM has 
>changed, the deployment planner can either select the same host in 
>which case the values are dynamically scaled up OR a different one in 
>which case the operation fails. In future if there is support for live 
>migration (provided HV supports it) then another option in the latter 
>case could be to migrate the VM first and then scale it up.
>
>I will start working on the FS and share it out sometime next week.
>
>Comments/suggestions?
>
>Thanks,
>Koushik

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Chip Childers <ch...@sungard.com>]

Nitin - 

Looking for your help here please!

-chip


On Tue, Feb 12, 2013 at 12:37:20PM +0530, Prashant Kumar Mishra wrote:
> 
> 
> -----Original Message-----
> From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com] 
> Sent: Tuesday, February 05, 2013 5:16 PM
> To: cloudstack-dev@incubator.apache.org
> Cc: Nitin Mehta
> Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
> 
> Need  reply  to complete  my Test cases.
> 
> -----Original Message-----
> From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com] 
> Sent: Thursday, January 24, 2013 12:26 PM
> To: cloudstack-dev@incubator.apache.org
> Cc: Nitin Mehta
> Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
> 
> Hi Nitin,
> I am planning to take the QA job for this feature. Have reviewed the functional spec, gone through community discussion  and have the following questions
> 
> 1-What is expected behavior of CS for Operating systems which do not support dynamic scaling . ?
>  
> 2-How much resources can be scaled up, is it limited by availability of resource on host .?
>  
> [Koushik Das ]
> "Having a range for CPU/RAM in compute offering is definitely another way of doing it. But creating the higher limit would be tricky. I am not sure if it is always known to users how much they want to scale up to at the time of deploying VM. Moreover if the higher limit is known then the VM can be deployed with that value itself. Also in case of having a range in the offering the usage part needs to be handled appropriately. Currently usage is purely based on the offering and individual values are not stored".
> [/Koushik Das]
> 
> it seems  its totally depend on service offering , please correct me if I am wrong. 
> 
> 3-  Scheduled snapshot of volumes during the operation .
> 
> [NITIN]
> For vmware, the entire vm is locked by HV and this can be an issue. I will leverage on current implementations for existing interactions like scheduled snapshots events during live migration and will replicate the same.
> [/NITIN]
> 
> Can you elaborate what is expected in case of VMware .
> 
> 4 - what is expected behavior in case of  powers off the vm during the operation .? is it different for different hypervisors.?
> 
> 5- what is expected in case of migration fails( In FS no description about this),  
>        -CS will  retry to migrate it again if yes how many time ?
>       - will it mark as a failure and can't  scale up(even resources are available in cluster) ?
> 
> 6- Apart from  "scaleVirtualMachine"    any other APIs are getting changed ?
> 
> 7-Scale down is allowed ? (still open issue in FS)
> 
> 8-Are we going to introduce custom compute offering (still open issue in FS) ?
> 
> 9- what are the guide line for upgrade  ?
> 
> 10-Any DB changes ?
> 
> 11- which Usage events are getting introduced for billing .?
> 
> 12-hypervisor support ,is it only for VMware (as per FS)  or its getting extended for XS/KVM also ?
> 
> Thanks
> Prashant Kumar Mishra
> 
> 
> -----Original Message-----
> From: Koushik Das [mailto:koushik.das@citrix.com] 
> Sent: Saturday, December 15, 2012 11:14 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
> 
> Currently CS supports changing CPU and RAM for stopped VM. This is achieved by changing compute offering of the VM (with new CPU and RAM values) and then starting it. I am planning to extend the same for running VM as well. Initially planning to do it for Vmware where CPU and RAM can be dynamically increased. Support of other HVs can also be added if they support increasing CPU/RAM.
> 
> Assuming that in the updated compute offering only CPU and RAM has changed, the deployment planner can either select the same host in which case the values are dynamically scaled up OR a different one in which case the operation fails. In future if there is support for live migration (provided HV supports it) then another option in the latter case could be to migrate the VM first and then scale it up.
> 
> I will start working on the FS and share it out sometime next week.
> 
> Comments/suggestions?
> 
> Thanks,
> Koushik
> 

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Nitin Mehta <Ni...@citrix.com>]

Apologize for the delayed response. Was involved in other issues.
Please find answers inline.

>
>-----Original Message-----
>From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com]
>Sent: Thursday, January 24, 2013 12:26 PM
>To: cloudstack-dev@incubator.apache.org
>Cc: Nitin Mehta
>Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs
>
>Hi Nitin,
>I am planning to take the QA job for this feature. Have reviewed the
>functional spec, gone through community discussion  and have the
>following questions
>
>1-What is expected behavior of CS for Operating systems which do not
>support dynamic scaling . ?

Will throw a not supported exception

> 
>2-How much resources can be scaled up, is it limited by availability of
>resource on host .?
> 
>[Koushik Das ]
>"Having a range for CPU/RAM in compute offering is definitely another way
>of doing it. But creating the higher limit would be tricky. I am not sure
>if it is always known to users how much they want to scale up to at the
>time of deploying VM. Moreover if the higher limit is known then the VM
>can be deployed with that value itself. Also in case of having a range in
>the offering the usage part needs to be handled appropriately. Currently
>usage is purely based on the offering and individual values are not
>stored".
>[/Koushik Das]

It is not limited by the resources on host but on the available capacity
in any of the host within the cluster.

>
>it seems  its totally depend on service offering , please correct me if I
>am wrong. 
>
>3-  Scheduled snapshot of volumes during the operation .
>
>[NITIN]
>For vmware, the entire vm is locked by HV and this can be an issue. I
>will leverage on current implementations for existing interactions like
>scheduled snapshots events during live migration and will replicate the
>same.
>[/NITIN]
>
>Can you elaborate what is expected in case of VMware .

What I mean is there is an existing functionality which is implemented the
same way. I will just do it the same way.

>
>4 - what is expected behavior in case of  powers off the vm during the
>operation .? is it different for different hypervisors.?

There is nothing much to do for powered off vms because we will just
update the DB. When the vm is started it will pick up these values from
the DB.
This functionality already exists.

>
>5- what is expected in case of migration fails( In FS no description
>about this),  
>       -CS will  retry to migrate it again if yes how many time ?
>      - will it mark as a failure and can't  scale up(even resources are
>available in cluster) ?

We will retry N (configurable) times and if unsuccessful we will throw an
exception to the user.

>
>6- Apart from  "scaleVirtualMachine"    any other APIs are getting
>changed ?

No 

>
>7-Scale down is allowed ? (still open issue in FS)

No for time being. 

>
>8-Are we going to introduce custom compute offering (still open issue in
>FS) ?

No for now 

>
>9- what are the guide line for upgrade  ?

There is nothing for upgrade because we do not introduce new values in DB.

>
>10-Any DB changes ?

See #9 above.

>
>11- which Usage events are getting introduced for billing .?

Will update the FS.

>
>12-hypervisor support ,is it only for VMware (as per FS)  or its getting
>extended for XS/KVM also ?

There are subtasks opened for XS and KVM. I am doing it only for Vmware.

>
>Thanks
>Prashant Kumar Mishra
>
>
>-----Original Message-----
>From: Koushik Das [mailto:koushik.das@citrix.com]
>Sent: Saturday, December 15, 2012 11:14 PM
>To: cloudstack-dev@incubator.apache.org
>Subject: [DISCUSS] Scaling up CPU and RAM for running VMs
>
>Currently CS supports changing CPU and RAM for stopped VM. This is
>achieved by changing compute offering of the VM (with new CPU and RAM
>values) and then starting it. I am planning to extend the same for
>running VM as well. Initially planning to do it for Vmware where CPU and
>RAM can be dynamically increased. Support of other HVs can also be added
>if they support increasing CPU/RAM.
>
>Assuming that in the updated compute offering only CPU and RAM has
>changed, the deployment planner can either select the same host in which
>case the values are dynamically scaled up OR a different one in which
>case the operation fails. In future if there is support for live
>migration (provided HV supports it) then another option in the latter
>case could be to migrate the VM first and then scale it up.
>
>I will start working on the FS and share it out sometime next week.
>
>Comments/suggestions?
>
>Thanks,
>Koushik

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Prashant Kumar Mishra <pr...@citrix.com>]

-----Original Message-----
From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com] 
Sent: Tuesday, February 05, 2013 5:16 PM
To: cloudstack-dev@incubator.apache.org
Cc: Nitin Mehta
Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs

Need  reply  to complete  my Test cases.

-----Original Message-----
From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com] 
Sent: Thursday, January 24, 2013 12:26 PM
To: cloudstack-dev@incubator.apache.org
Cc: Nitin Mehta
Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs

Hi Nitin,
I am planning to take the QA job for this feature. Have reviewed the functional spec, gone through community discussion  and have the following questions

1-What is expected behavior of CS for Operating systems which do not support dynamic scaling . ?
 
2-How much resources can be scaled up, is it limited by availability of resource on host .?
 
[Koushik Das ]
"Having a range for CPU/RAM in compute offering is definitely another way of doing it. But creating the higher limit would be tricky. I am not sure if it is always known to users how much they want to scale up to at the time of deploying VM. Moreover if the higher limit is known then the VM can be deployed with that value itself. Also in case of having a range in the offering the usage part needs to be handled appropriately. Currently usage is purely based on the offering and individual values are not stored".
[/Koushik Das]

it seems  its totally depend on service offering , please correct me if I am wrong. 

3-  Scheduled snapshot of volumes during the operation .

[NITIN]
For vmware, the entire vm is locked by HV and this can be an issue. I will leverage on current implementations for existing interactions like scheduled snapshots events during live migration and will replicate the same.
[/NITIN]

Can you elaborate what is expected in case of VMware .

4 - what is expected behavior in case of  powers off the vm during the operation .? is it different for different hypervisors.?

5- what is expected in case of migration fails( In FS no description about this),  
       -CS will  retry to migrate it again if yes how many time ?
      - will it mark as a failure and can't  scale up(even resources are available in cluster) ?

6- Apart from  "scaleVirtualMachine"    any other APIs are getting changed ?

7-Scale down is allowed ? (still open issue in FS)

8-Are we going to introduce custom compute offering (still open issue in FS) ?

9- what are the guide line for upgrade  ?

10-Any DB changes ?

11- which Usage events are getting introduced for billing .?

12-hypervisor support ,is it only for VMware (as per FS)  or its getting extended for XS/KVM also ?

Thanks
Prashant Kumar Mishra


-----Original Message-----
From: Koushik Das [mailto:koushik.das@citrix.com] 
Sent: Saturday, December 15, 2012 11:14 PM
To: cloudstack-dev@incubator.apache.org
Subject: [DISCUSS] Scaling up CPU and RAM for running VMs

Currently CS supports changing CPU and RAM for stopped VM. This is achieved by changing compute offering of the VM (with new CPU and RAM values) and then starting it. I am planning to extend the same for running VM as well. Initially planning to do it for Vmware where CPU and RAM can be dynamically increased. Support of other HVs can also be added if they support increasing CPU/RAM.

Assuming that in the updated compute offering only CPU and RAM has changed, the deployment planner can either select the same host in which case the values are dynamically scaled up OR a different one in which case the operation fails. In future if there is support for live migration (provided HV supports it) then another option in the latter case could be to migrate the VM first and then scale it up.

I will start working on the FS and share it out sometime next week.

Comments/suggestions?

Thanks,
Koushik

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Prashant Kumar Mishra <pr...@citrix.com>]

Need  reply  to complete  my Test cases.

-----Original Message-----
From: Prashant Kumar Mishra [mailto:prashantkumar.mishra@citrix.com] 
Sent: Thursday, January 24, 2013 12:26 PM
To: cloudstack-dev@incubator.apache.org
Cc: Nitin Mehta
Subject: RE: [DISCUSS] Scaling up CPU and RAM for running VMs

Hi Nitin,
I am planning to take the QA job for this feature. Have reviewed the functional spec, gone through community discussion  and have the following questions

1-What is expected behavior of CS for Operating systems which do not support dynamic scaling . ?
 
2-How much resources can be scaled up, is it limited by availability of resource on host .?
 
[Koushik Das ]
"Having a range for CPU/RAM in compute offering is definitely another way of doing it. But creating the higher limit would be tricky. I am not sure if it is always known to users how much they want to scale up to at the time of deploying VM. Moreover if the higher limit is known then the VM can be deployed with that value itself. Also in case of having a range in the offering the usage part needs to be handled appropriately. Currently usage is purely based on the offering and individual values are not stored".
[/Koushik Das]

it seems  its totally depend on service offering , please correct me if I am wrong. 

3-  Scheduled snapshot of volumes during the operation .

[NITIN]
For vmware, the entire vm is locked by HV and this can be an issue. I will leverage on current implementations for existing interactions like scheduled snapshots events during live migration and will replicate the same.
[/NITIN]

Can you elaborate what is expected in case of VMware .

4 - what is expected behavior in case of  powers off the vm during the operation .? is it different for different hypervisors.?

5- what is expected in case of migration fails( In FS no description about this),  
       -CS will  retry to migrate it again if yes how many time ?
      - will it mark as a failure and can't  scale up(even resources are available in cluster) ?

6- Apart from  "scaleVirtualMachine"    any other APIs are getting changed ?

7-Scale down is allowed ? (still open issue in FS)

8-Are we going to introduce custom compute offering (still open issue in FS) ?

9- what are the guide line for upgrade  ?

10-Any DB changes ?

11- which Usage events are getting introduced for billing .?

12-hypervisor support ,is it only for VMware (as per FS)  or its getting extended for XS/KVM also ?

Thanks
Prashant Kumar Mishra


-----Original Message-----
From: Koushik Das [mailto:koushik.das@citrix.com] 
Sent: Saturday, December 15, 2012 11:14 PM
To: cloudstack-dev@incubator.apache.org
Subject: [DISCUSS] Scaling up CPU and RAM for running VMs

Currently CS supports changing CPU and RAM for stopped VM. This is achieved by changing compute offering of the VM (with new CPU and RAM values) and then starting it. I am planning to extend the same for running VM as well. Initially planning to do it for Vmware where CPU and RAM can be dynamically increased. Support of other HVs can also be added if they support increasing CPU/RAM.

Assuming that in the updated compute offering only CPU and RAM has changed, the deployment planner can either select the same host in which case the values are dynamically scaled up OR a different one in which case the operation fails. In future if there is support for live migration (provided HV supports it) then another option in the latter case could be to migrate the VM first and then scale it up.

I will start working on the FS and share it out sometime next week.

Comments/suggestions?

Thanks,
Koushik

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Prashant Kumar Mishra <pr...@citrix.com>]

Hi Nitin,
I am planning to take the QA job for this feature. Have reviewed the functional spec, gone through community discussion  and have the following questions

1-What is expected behavior of CS for Operating systems which do not support dynamic scaling . ?
 
2-How much resources can be scaled up, is it limited by availability of resource on host .?
 
[Koushik Das ]
"Having a range for CPU/RAM in compute offering is definitely another way of doing it. But creating the higher limit would be tricky. I am not sure if it is always known to users how much they want to scale up to at the time of deploying VM. Moreover if the higher limit is known then the VM can be deployed with that value itself. Also in case of having a range in the offering the usage part needs to be handled appropriately. Currently usage is purely based on the offering and individual values are not stored".
[/Koushik Das]

it seems  its totally depend on service offering , please correct me if I am wrong. 

3-  Scheduled snapshot of volumes during the operation .

[NITIN]
For vmware, the entire vm is locked by HV and this can be an issue. I will leverage on current implementations for existing interactions like scheduled snapshots events during live migration and will replicate the same.
[/NITIN]

Can you elaborate what is expected in case of VMware .

4 - what is expected behavior in case of  powers off the vm during the operation .? is it different for different hypervisors.?

5- what is expected in case of migration fails( In FS no description about this),  
       -CS will  retry to migrate it again if yes how many time ?
      - will it mark as a failure and can't  scale up(even resources are available in cluster) ?

6- Apart from  "scaleVirtualMachine"    any other APIs are getting changed ?

7-Scale down is allowed ? (still open issue in FS)

8-Are we going to introduce custom compute offering (still open issue in FS) ?

9- what are the guide line for upgrade  ?

10-Any DB changes ?

11- which Usage events are getting introduced for billing .?

12-hypervisor support ,is it only for VMware (as per FS)  or its getting extended for XS/KVM also ?

Thanks
Prashant Kumar Mishra


-----Original Message-----
From: Koushik Das [mailto:koushik.das@citrix.com] 
Sent: Saturday, December 15, 2012 11:14 PM
To: cloudstack-dev@incubator.apache.org
Subject: [DISCUSS] Scaling up CPU and RAM for running VMs

Currently CS supports changing CPU and RAM for stopped VM. This is achieved by changing compute offering of the VM (with new CPU and RAM values) and then starting it. I am planning to extend the same for running VM as well. Initially planning to do it for Vmware where CPU and RAM can be dynamically increased. Support of other HVs can also be added if they support increasing CPU/RAM.

Assuming that in the updated compute offering only CPU and RAM has changed, the deployment planner can either select the same host in which case the values are dynamically scaled up OR a different one in which case the operation fails. In future if there is support for live migration (provided HV supports it) then another option in the latter case could be to migrate the VM first and then scale it up.

I will start working on the FS and share it out sometime next week.

Comments/suggestions?

Thanks,
Koushik

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Mice Xia <we...@gmail.com>]

[David]
I am worried about the effect of this on usage statistics, but still
sounds very useful.
[/David]

Currently EVENT_VM_UPGRADE is already handled in usage server, it
records upgraded service offering into usage DB (though i cant find
any codes that persist this eventType). Maybe this can be re-used for
this feature if we only allow upgrade/downgrade between static service
offerings.

[Nitin]
Mice - Great point. But, I think when the vm is scaled up, allocators
need to be wise enough to figure out if it could be adjusted on the
same host or need live migration. I think atleast XS and Vmware
support live migration (across clusters as well) now so dynamic
scaling would have less probability of failing, but this could be an
expensive operation because migrating volumes across clusters will be
resource intensive.
[/Nitin]

yeah sometimes migration is unavoidable, memory overprovisioning is
somewhat irrelavant to this topic.

Regards
Mice

RE: [DISCUSS] Scaling up CPU and RAM for running VMs

[Koushik Das <ko...@citrix.com>]

See inline

> -----Original Message-----
> From: David Nalley [mailto:david@gnsa.us]
> Sent: Sunday, December 16, 2012 3:56 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [DISCUSS] Scaling up CPU and RAM for running VMs
> 
> On Sat, Dec 15, 2012 at 12:43 PM, Koushik Das <ko...@citrix.com>
> wrote:
> > Currently CS supports changing CPU and RAM for stopped VM. This is
> achieved by changing compute offering of the VM (with new CPU and RAM
> values) and then starting it. I am planning to extend the same for running VM
> as well. Initially planning to do it for Vmware where CPU and RAM can be
> dynamically increased. Support of other HVs can also be added if they
> support increasing CPU/RAM.
> >
> > Assuming that in the updated compute offering only CPU and RAM has
> changed, the deployment planner can either select the same host in which
> case the values are dynamically scaled up OR a different one in which case
> the operation fails. In future if there is support for live migration (provided
> HV supports it) then another option in the latter case could be to migrate the
> VM first and then scale it up.
> >
> > I will start working on the FS and share it out sometime next week.
> 
> Thanks for starting this conversation early in the process, Koushik.
> 
> I'd want to make just 'how much' resources can be scaled up a configuration
> option. Is 2x reasonable? probably, is 50x reasonable, probably not.
> Or actually - thinking about this - perhaps this becomes part of the service
> offering. You can start at 2cores/4gb but have a place to scale up to without
> restart, but you know from the outset exactly how far that is. Actually - since
> you can't scale it back down, that makes even more sense to me. (I know you
> can't scale back down in KVM, not so sure about VMware) I am worried
> about the effect of this on usage statistics, but still sounds very useful.
> 

Having a range for CPU/RAM in compute offering is definitely another way of doing it. But creating the higher limit would be tricky. I am not sure if it is always known to users how much they want to scale up to at the time of deploying VM. Moreover if the higher limit is known then the VM can be deployed with that value itself. Also in case of having a range in the offering the usage part needs to be handled appropriately. Currently usage is purely based on the offering and individual values are not stored.

> Maybe you solve the allocation problem by actually allocating for the max
> allocatable - of course if that goes into usage metrics, and then on to billing
> perhaps it isn't as attractive.
> 
> --David

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[Marcus Sorensen <sh...@gmail.com>]

You can scale down memory in KVM, but not CPU without a reboot. I'd like to
see this implemented as the ability to jump between service offerings
without reboot, and perhaps have a custom service offering that allows for
any custom number to be put in. That way people who utilize cloudstack can
choose between making only certain offerings/combinations available
(control the scale amount as David said), or opening it up for full
customization.


On Sat, Dec 15, 2012 at 3:25 PM, David Nalley <da...@gnsa.us> wrote:

> On Sat, Dec 15, 2012 at 12:43 PM, Koushik Das <ko...@citrix.com>
> wrote:
> > Currently CS supports changing CPU and RAM for stopped VM. This is
> achieved by changing compute offering of the VM (with new CPU and RAM
> values) and then starting it. I am planning to extend the same for running
> VM as well. Initially planning to do it for Vmware where CPU and RAM can be
> dynamically increased. Support of other HVs can also be added if they
> support increasing CPU/RAM.
> >
> > Assuming that in the updated compute offering only CPU and RAM has
> changed, the deployment planner can either select the same host in which
> case the values are dynamically scaled up OR a different one in which case
> the operation fails. In future if there is support for live migration
> (provided HV supports it) then another option in the latter case could be
> to migrate the VM first and then scale it up.
> >
> > I will start working on the FS and share it out sometime next week.
>
> Thanks for starting this conversation early in the process, Koushik.
>
> I'd want to make just 'how much' resources can be scaled up a
> configuration option. Is 2x reasonable? probably, is 50x reasonable,
> probably not.
> Or actually - thinking about this - perhaps this becomes part of the
> service offering. You can start at 2cores/4gb but have a place to
> scale up to without restart, but you know from the outset exactly how
> far that is. Actually - since you can't scale it back down, that makes
> even more sense to me. (I know you can't scale back down in KVM, not
> so sure about VMware) I am worried about the effect of this on usage
> statistics, but still sounds very useful.
>
> Maybe you solve the allocation problem by actually allocating for the
> max allocatable - of course if that goes into usage metrics, and then
> on to billing perhaps it isn't as attractive.
>
> --David
>

Re: [DISCUSS] Scaling up CPU and RAM for running VMs

[David Nalley <da...@gnsa.us>]

On Sat, Dec 15, 2012 at 12:43 PM, Koushik Das <ko...@citrix.com> wrote:
> Currently CS supports changing CPU and RAM for stopped VM. This is achieved by changing compute offering of the VM (with new CPU and RAM values) and then starting it. I am planning to extend the same for running VM as well. Initially planning to do it for Vmware where CPU and RAM can be dynamically increased. Support of other HVs can also be added if they support increasing CPU/RAM.
>
> Assuming that in the updated compute offering only CPU and RAM has changed, the deployment planner can either select the same host in which case the values are dynamically scaled up OR a different one in which case the operation fails. In future if there is support for live migration (provided HV supports it) then another option in the latter case could be to migrate the VM first and then scale it up.
>
> I will start working on the FS and share it out sometime next week.

Thanks for starting this conversation early in the process, Koushik.

I'd want to make just 'how much' resources can be scaled up a
configuration option. Is 2x reasonable? probably, is 50x reasonable,
probably not.
Or actually - thinking about this - perhaps this becomes part of the
service offering. You can start at 2cores/4gb but have a place to
scale up to without restart, but you know from the outset exactly how
far that is. Actually - since you can't scale it back down, that makes
even more sense to me. (I know you can't scale back down in KVM, not
so sure about VMware) I am worried about the effect of this on usage
statistics, but still sounds very useful.

Maybe you solve the allocation problem by actually allocating for the
max allocatable - of course if that goes into usage metrics, and then
on to billing perhaps it isn't as attractive.

--David