You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Mike Tutkowski <mi...@solidfire.com> on 2014/07/14 23:23:36 UTC
Job Queue Question
Hi,
I have a question about this logic (related to resizing a volume):
AsyncJobExecutionContext jobContext =
AsyncJobExecutionContext.getCurrentExecutionContext();
if (!VmJobEnabled.value() ||
jobContext.isJobDispatchedBy(VmWorkConstants.VM_WORK_JOB_DISPATCHER)) {
// avoid re-entrance
VmWorkJobVO placeHolder = null;
if (VmJobEnabled.value()) {
placeHolder = createPlaceHolderWork(userVm.getId());
}
try {
return orchestrateResizeVolume(volume.getId(),
currentSize, newSize,
newDiskOffering != null ?
cmd.getNewDiskOfferingId() : null, shrinkOk);
} finally {
if (VmJobEnabled.value()) {
_workJobDao.expunge(placeHolder.getId());
}
}
} else {
Outcome<Volume> outcome =
resizeVolumeThroughJobQueue(userVm.getId(), volume.getId(), currentSize,
newSize,
newDiskOffering != null ?
cmd.getNewDiskOfferingId() : null, shrinkOk);
Why would one resize the volume via the job queue versus the other path?
Thanks!
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
<http://solidfire.com/solution/overview/?video=play>*™*
Re: Job Queue Question
Posted by Mike Tutkowski <mi...@solidfire.com>.
Yes, I would be interested in this, as well. Thanks
On Mon, Jul 14, 2014 at 11:07 PM, Amit Das <am...@cloudbyte.com> wrote:
> >
> > this is part of the refactoring principal from new VM state
> > synchronization framework.
>
>
> What is the corresponding cloudstack namespace for above framework ?
>
> Is there any documentation available w.r.t synchronization framework ?
>
>
>
> Regards,
> Amit
> *CloudByte Inc.* <http://www.cloudbyte.com/>
>
>
> On Tue, Jul 15, 2014 at 4:36 AM, Kelven Yang <ke...@citrix.com>
> wrote:
>
> > Yes, this is part of the refactoring principal from new VM state
> > synchronization framework. We serialize operations happening to the VM,
> > for each individual operation flow, it has a clear context to perform its
> > task without worrying about disruption from other flows, therefore create
> > a loosely couple way of doing orchestration.
> >
> > We used to synchronize based on DB lock table, the drawback of this
> > approach is that not only we may have multiple flows to continuously pull
> > database, but also that we have very few control over the orchestration
> > flows and manage system load accordingly.
> >
> > The new way can let us explore possibility of things like having better
> > visibility on orchestration flows, canceling job, load throttling etc,
> and
> > these abilities can be achieved through improving the job queue facility
> > itself without impact to individual orchestration flows.
> >
> > Kelven
> >
> > On 7/14/14, 3:13 PM, "Mike Tutkowski" <mi...@solidfire.com>
> > wrote:
> >
> > >I see...so, in case, for example, this API call is being executed in
> > >multiple places at the same time, we are making sure we send the
> commands
> > >to the VM in a serial fashion.
> > >
> > >
> > >On Mon, Jul 14, 2014 at 4:09 PM, Kelven Yang <ke...@citrix.com>
> > >wrote:
> > >
> > >> Mike,
> > >>
> > >> This is related to serializing activities to the VM. When VM has
> > >>multiple
> > >> disks and volume-semantic API could create situations that there exist
> > >> multiple volume operations happening at the same time on the same VM.
> > >>
> > >> Kelven
> > >>
> > >> On 7/14/14, 2:23 PM, "Mike Tutkowski" <mi...@solidfire.com>
> > >> wrote:
> > >>
> > >> >Hi,
> > >> >
> > >> >I have a question about this logic (related to resizing a volume):
> > >> >
> > >> > AsyncJobExecutionContext jobContext =
> > >> >AsyncJobExecutionContext.getCurrentExecutionContext();
> > >> >
> > >> >
> > >> > if (!VmJobEnabled.value() ||
> > >>
> >jobContext.isJobDispatchedBy(VmWorkConstants.VM_WORK_JOB_DISPATCHER)) {
> > >> >
> > >> > // avoid re-entrance
> > >> >
> > >> >
> > >> > VmWorkJobVO placeHolder = null;
> > >> >
> > >> >
> > >> > if (VmJobEnabled.value()) {
> > >> >
> > >> > placeHolder =
> > >>createPlaceHolderWork(userVm.getId());
> > >> >
> > >> > }
> > >> >
> > >> >
> > >> > try {
> > >> >
> > >> > return orchestrateResizeVolume(volume.getId(),
> > >> >currentSize, newSize,
> > >> >
> > >> > newDiskOffering != null ?
> > >> >cmd.getNewDiskOfferingId() : null, shrinkOk);
> > >> >
> > >> > } finally {
> > >> >
> > >> > if (VmJobEnabled.value()) {
> > >> >
> > >> > _workJobDao.expunge(placeHolder.getId());
> > >> >
> > >> > }
> > >> >
> > >> > }
> > >> >
> > >> >
> > >> > } else {
> > >> >
> > >> > Outcome<Volume> outcome =
> > >> >resizeVolumeThroughJobQueue(userVm.getId(), volume.getId(),
> > >>currentSize,
> > >> >newSize,
> > >> >
> > >> > newDiskOffering != null ?
> > >> >cmd.getNewDiskOfferingId() : null, shrinkOk);
> > >> >
> > >> >
> > >> >Why would one resize the volume via the job queue versus the other
> > >>path?
> > >> >
> > >> >Thanks!
> > >> >--
> > >> >*Mike Tutkowski*
> > >> >*Senior CloudStack Developer, SolidFire Inc.*
> > >> >e: mike.tutkowski@solidfire.com
> > >> >o: 303.746.7302
> > >> >Advancing the way the world uses the cloud
> > >> ><http://solidfire.com/solution/overview/?video=play>* *
> > >>
> > >>
> > >
> > >
> > >--
> > >*Mike Tutkowski*
> > >*Senior CloudStack Developer, SolidFire Inc.*
> > >e: mike.tutkowski@solidfire.com
> > >o: 303.746.7302
> > >Advancing the way the world uses the cloud
> > ><http://solidfire.com/solution/overview/?video=play>* *
> >
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
<http://solidfire.com/solution/overview/?video=play>*™*
Re: Job Queue Question
Posted by Amit Das <am...@cloudbyte.com>.
>
> this is part of the refactoring principal from new VM state
> synchronization framework.
What is the corresponding cloudstack namespace for above framework ?
Is there any documentation available w.r.t synchronization framework ?
Regards,
Amit
*CloudByte Inc.* <http://www.cloudbyte.com/>
On Tue, Jul 15, 2014 at 4:36 AM, Kelven Yang <ke...@citrix.com> wrote:
> Yes, this is part of the refactoring principal from new VM state
> synchronization framework. We serialize operations happening to the VM,
> for each individual operation flow, it has a clear context to perform its
> task without worrying about disruption from other flows, therefore create
> a loosely couple way of doing orchestration.
>
> We used to synchronize based on DB lock table, the drawback of this
> approach is that not only we may have multiple flows to continuously pull
> database, but also that we have very few control over the orchestration
> flows and manage system load accordingly.
>
> The new way can let us explore possibility of things like having better
> visibility on orchestration flows, canceling job, load throttling etc, and
> these abilities can be achieved through improving the job queue facility
> itself without impact to individual orchestration flows.
>
> Kelven
>
> On 7/14/14, 3:13 PM, "Mike Tutkowski" <mi...@solidfire.com>
> wrote:
>
> >I see...so, in case, for example, this API call is being executed in
> >multiple places at the same time, we are making sure we send the commands
> >to the VM in a serial fashion.
> >
> >
> >On Mon, Jul 14, 2014 at 4:09 PM, Kelven Yang <ke...@citrix.com>
> >wrote:
> >
> >> Mike,
> >>
> >> This is related to serializing activities to the VM. When VM has
> >>multiple
> >> disks and volume-semantic API could create situations that there exist
> >> multiple volume operations happening at the same time on the same VM.
> >>
> >> Kelven
> >>
> >> On 7/14/14, 2:23 PM, "Mike Tutkowski" <mi...@solidfire.com>
> >> wrote:
> >>
> >> >Hi,
> >> >
> >> >I have a question about this logic (related to resizing a volume):
> >> >
> >> > AsyncJobExecutionContext jobContext =
> >> >AsyncJobExecutionContext.getCurrentExecutionContext();
> >> >
> >> >
> >> > if (!VmJobEnabled.value() ||
> >> >jobContext.isJobDispatchedBy(VmWorkConstants.VM_WORK_JOB_DISPATCHER)) {
> >> >
> >> > // avoid re-entrance
> >> >
> >> >
> >> > VmWorkJobVO placeHolder = null;
> >> >
> >> >
> >> > if (VmJobEnabled.value()) {
> >> >
> >> > placeHolder =
> >>createPlaceHolderWork(userVm.getId());
> >> >
> >> > }
> >> >
> >> >
> >> > try {
> >> >
> >> > return orchestrateResizeVolume(volume.getId(),
> >> >currentSize, newSize,
> >> >
> >> > newDiskOffering != null ?
> >> >cmd.getNewDiskOfferingId() : null, shrinkOk);
> >> >
> >> > } finally {
> >> >
> >> > if (VmJobEnabled.value()) {
> >> >
> >> > _workJobDao.expunge(placeHolder.getId());
> >> >
> >> > }
> >> >
> >> > }
> >> >
> >> >
> >> > } else {
> >> >
> >> > Outcome<Volume> outcome =
> >> >resizeVolumeThroughJobQueue(userVm.getId(), volume.getId(),
> >>currentSize,
> >> >newSize,
> >> >
> >> > newDiskOffering != null ?
> >> >cmd.getNewDiskOfferingId() : null, shrinkOk);
> >> >
> >> >
> >> >Why would one resize the volume via the job queue versus the other
> >>path?
> >> >
> >> >Thanks!
> >> >--
> >> >*Mike Tutkowski*
> >> >*Senior CloudStack Developer, SolidFire Inc.*
> >> >e: mike.tutkowski@solidfire.com
> >> >o: 303.746.7302
> >> >Advancing the way the world uses the cloud
> >> ><http://solidfire.com/solution/overview/?video=play>* *
> >>
> >>
> >
> >
> >--
> >*Mike Tutkowski*
> >*Senior CloudStack Developer, SolidFire Inc.*
> >e: mike.tutkowski@solidfire.com
> >o: 303.746.7302
> >Advancing the way the world uses the cloud
> ><http://solidfire.com/solution/overview/?video=play>* *
>
>
Re: Job Queue Question
Posted by Kelven Yang <ke...@citrix.com>.
Yes, this is part of the refactoring principal from new VM state
synchronization framework. We serialize operations happening to the VM,
for each individual operation flow, it has a clear context to perform its
task without worrying about disruption from other flows, therefore create
a loosely couple way of doing orchestration.
We used to synchronize based on DB lock table, the drawback of this
approach is that not only we may have multiple flows to continuously pull
database, but also that we have very few control over the orchestration
flows and manage system load accordingly.
The new way can let us explore possibility of things like having better
visibility on orchestration flows, canceling job, load throttling etc, and
these abilities can be achieved through improving the job queue facility
itself without impact to individual orchestration flows.
Kelven
On 7/14/14, 3:13 PM, "Mike Tutkowski" <mi...@solidfire.com> wrote:
>I see...so, in case, for example, this API call is being executed in
>multiple places at the same time, we are making sure we send the commands
>to the VM in a serial fashion.
>
>
>On Mon, Jul 14, 2014 at 4:09 PM, Kelven Yang <ke...@citrix.com>
>wrote:
>
>> Mike,
>>
>> This is related to serializing activities to the VM. When VM has
>>multiple
>> disks and volume-semantic API could create situations that there exist
>> multiple volume operations happening at the same time on the same VM.
>>
>> Kelven
>>
>> On 7/14/14, 2:23 PM, "Mike Tutkowski" <mi...@solidfire.com>
>> wrote:
>>
>> >Hi,
>> >
>> >I have a question about this logic (related to resizing a volume):
>> >
>> > AsyncJobExecutionContext jobContext =
>> >AsyncJobExecutionContext.getCurrentExecutionContext();
>> >
>> >
>> > if (!VmJobEnabled.value() ||
>> >jobContext.isJobDispatchedBy(VmWorkConstants.VM_WORK_JOB_DISPATCHER)) {
>> >
>> > // avoid re-entrance
>> >
>> >
>> > VmWorkJobVO placeHolder = null;
>> >
>> >
>> > if (VmJobEnabled.value()) {
>> >
>> > placeHolder =
>>createPlaceHolderWork(userVm.getId());
>> >
>> > }
>> >
>> >
>> > try {
>> >
>> > return orchestrateResizeVolume(volume.getId(),
>> >currentSize, newSize,
>> >
>> > newDiskOffering != null ?
>> >cmd.getNewDiskOfferingId() : null, shrinkOk);
>> >
>> > } finally {
>> >
>> > if (VmJobEnabled.value()) {
>> >
>> > _workJobDao.expunge(placeHolder.getId());
>> >
>> > }
>> >
>> > }
>> >
>> >
>> > } else {
>> >
>> > Outcome<Volume> outcome =
>> >resizeVolumeThroughJobQueue(userVm.getId(), volume.getId(),
>>currentSize,
>> >newSize,
>> >
>> > newDiskOffering != null ?
>> >cmd.getNewDiskOfferingId() : null, shrinkOk);
>> >
>> >
>> >Why would one resize the volume via the job queue versus the other
>>path?
>> >
>> >Thanks!
>> >--
>> >*Mike Tutkowski*
>> >*Senior CloudStack Developer, SolidFire Inc.*
>> >e: mike.tutkowski@solidfire.com
>> >o: 303.746.7302
>> >Advancing the way the world uses the cloud
>> ><http://solidfire.com/solution/overview/?video=play>* *
>>
>>
>
>
>--
>*Mike Tutkowski*
>*Senior CloudStack Developer, SolidFire Inc.*
>e: mike.tutkowski@solidfire.com
>o: 303.746.7302
>Advancing the way the world uses the cloud
><http://solidfire.com/solution/overview/?video=play>**
Re: Job Queue Question
Posted by Mike Tutkowski <mi...@solidfire.com>.
I see...so, in case, for example, this API call is being executed in
multiple places at the same time, we are making sure we send the commands
to the VM in a serial fashion.
On Mon, Jul 14, 2014 at 4:09 PM, Kelven Yang <ke...@citrix.com> wrote:
> Mike,
>
> This is related to serializing activities to the VM. When VM has multiple
> disks and volume-semantic API could create situations that there exist
> multiple volume operations happening at the same time on the same VM.
>
> Kelven
>
> On 7/14/14, 2:23 PM, "Mike Tutkowski" <mi...@solidfire.com>
> wrote:
>
> >Hi,
> >
> >I have a question about this logic (related to resizing a volume):
> >
> > AsyncJobExecutionContext jobContext =
> >AsyncJobExecutionContext.getCurrentExecutionContext();
> >
> >
> > if (!VmJobEnabled.value() ||
> >jobContext.isJobDispatchedBy(VmWorkConstants.VM_WORK_JOB_DISPATCHER)) {
> >
> > // avoid re-entrance
> >
> >
> > VmWorkJobVO placeHolder = null;
> >
> >
> > if (VmJobEnabled.value()) {
> >
> > placeHolder = createPlaceHolderWork(userVm.getId());
> >
> > }
> >
> >
> > try {
> >
> > return orchestrateResizeVolume(volume.getId(),
> >currentSize, newSize,
> >
> > newDiskOffering != null ?
> >cmd.getNewDiskOfferingId() : null, shrinkOk);
> >
> > } finally {
> >
> > if (VmJobEnabled.value()) {
> >
> > _workJobDao.expunge(placeHolder.getId());
> >
> > }
> >
> > }
> >
> >
> > } else {
> >
> > Outcome<Volume> outcome =
> >resizeVolumeThroughJobQueue(userVm.getId(), volume.getId(), currentSize,
> >newSize,
> >
> > newDiskOffering != null ?
> >cmd.getNewDiskOfferingId() : null, shrinkOk);
> >
> >
> >Why would one resize the volume via the job queue versus the other path?
> >
> >Thanks!
> >--
> >*Mike Tutkowski*
> >*Senior CloudStack Developer, SolidFire Inc.*
> >e: mike.tutkowski@solidfire.com
> >o: 303.746.7302
> >Advancing the way the world uses the cloud
> ><http://solidfire.com/solution/overview/?video=play>* *
>
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkowski@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
<http://solidfire.com/solution/overview/?video=play>*™*
Re: Job Queue Question
Posted by Kelven Yang <ke...@citrix.com>.
Mike,
This is related to serializing activities to the VM. When VM has multiple
disks and volume-semantic API could create situations that there exist
multiple volume operations happening at the same time on the same VM.
Kelven
On 7/14/14, 2:23 PM, "Mike Tutkowski" <mi...@solidfire.com> wrote:
>Hi,
>
>I have a question about this logic (related to resizing a volume):
>
> AsyncJobExecutionContext jobContext =
>AsyncJobExecutionContext.getCurrentExecutionContext();
>
>
> if (!VmJobEnabled.value() ||
>jobContext.isJobDispatchedBy(VmWorkConstants.VM_WORK_JOB_DISPATCHER)) {
>
> // avoid re-entrance
>
>
> VmWorkJobVO placeHolder = null;
>
>
> if (VmJobEnabled.value()) {
>
> placeHolder = createPlaceHolderWork(userVm.getId());
>
> }
>
>
> try {
>
> return orchestrateResizeVolume(volume.getId(),
>currentSize, newSize,
>
> newDiskOffering != null ?
>cmd.getNewDiskOfferingId() : null, shrinkOk);
>
> } finally {
>
> if (VmJobEnabled.value()) {
>
> _workJobDao.expunge(placeHolder.getId());
>
> }
>
> }
>
>
> } else {
>
> Outcome<Volume> outcome =
>resizeVolumeThroughJobQueue(userVm.getId(), volume.getId(), currentSize,
>newSize,
>
> newDiskOffering != null ?
>cmd.getNewDiskOfferingId() : null, shrinkOk);
>
>
>Why would one resize the volume via the job queue versus the other path?
>
>Thanks!
>--
>*Mike Tutkowski*
>*Senior CloudStack Developer, SolidFire Inc.*
>e: mike.tutkowski@solidfire.com
>o: 303.746.7302
>Advancing the way the world uses the cloud
><http://solidfire.com/solution/overview/?video=play>**