You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@vcl.apache.org by Aaron Peeler <fa...@ncsu.edu> on 2012/10/18 16:20:51 UTC
Re: [jira] [Commented] (VCL-630) currentimage.txt name may conflict
with imagerevision name, causing unnecessary reloads
I can work on this one.
My thoughts to proceed, first convert OS.pm:sub
get_currentimage_txt_contents to return a hash of the contents of
currentimage.txt then update anything that currently depends on it
which I think is only OS.pm:sub get_current_image_name
Update get_current_image_name to return optional values, imageid,
imagerevision_id, imagename, etc. The default would be
imagerevision_id.
Then in update any provisioning modules node_status routine to start using
$self->os->get_current_image_name()
based on the desired return value, then node_status or other routines
can then compare what is loaded vs what is requested.
and lastest remove the old utils::_getcurrentimage routine
This would allow for changes in the name like shorter and less strict
naming of the internal imagenames.
Thoughts or comments before I proceed?
Aaron P.
On Fri, Sep 21, 2012 at 11:07 AM, Aaron Peeler (JIRA) <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/VCL-630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460545#comment-13460545 ]
>
> Aaron Peeler commented on VCL-630:
> ----------------------------------
>
> follow-up - there is a get_currentimage_txt_contents in the OS.pm module already. I think the best path would be to create a OS::get_currentimage_imagerevision_id routine which parses the contents and returns the imagerevisionid on the loaded node. Then update the provisioning module's node_status routines to use this instead.
>
>> currentimage.txt name may conflict with imagerevision name, causing unnecessary reloads
>> ---------------------------------------------------------------------------------------
>>
>> Key: VCL-630
>> URL: https://issues.apache.org/jira/browse/VCL-630
>> Project: VCL
>> Issue Type: Bug
>> Components: vcld (backend)
>> Affects Versions: 2.3
>> Environment: accessing the first revision of an image on a clustered (vCenter) VMware host. This only affects images whose name (cleaned of non-alphanumeric characters) exceeds 29 characters.
>> Reporter: Aaron Coburn
>> Assignee: Aaron Coburn
>> Fix For: 2.4, 2.3.1
>>
>>
>> A few words to provide some background: when the VCL uses a clustered (vCenter) VMware host, one cannot simply copy virtual disks as part of the image capture process; instead, the VM is cloned. This, however, exposed an apparently undocumented feature of VMware in that virtual disk names exceeding 29 characters are silently truncated. This creates significant problems for the VCL. The current solution (for 2.3) is to update the image name in the database after successfully capturing an image. This way, the VCL knows where to find the virtual disk in the VMware datastore.
>> The problem with this approach is that this change isn't registered with the system until after the image has been captured. At this point, however, the currentimage.txt file has already been written to the disk and if the name changes ex post facto, then the problem simply re-emerges in a different form when a user makes a reservation for the image.
>> Assuming the VM is available and otherwise ready to grant an immediate reservation, the reclaim.pm:process() subroutine checks the VM name listed in the currentimage.txt file and finds a conflict in the name listed on the OS and the name in the database.
>> An immediate solution for users is to simply make a new revision of the same image. That second revision will have a shortened name in both currentimage.txt and the imagerevision table.
>> There are two ways that this could be correctly solved in the perl code: either by updating the image name earlier in the provisioning module's capture() subroutine or by extracting other information from currentimage.txt, such as imagerevision_id.
>> While the first approach would be easier to implement, the second approach seems to be more architecturally sound. That is, rather than relying on a comparison of two strings, the test would be between two integers, one of which is the primary key of the database table in question.
>> Any thoughts on the best way to proceed?
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
--
Aaron Peeler
Program Manager
Virtual Computing Lab
NC State University
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.
Re: [jira] [Commented] (VCL-630) currentimage.txt name may conflict
with imagerevision name, causing unnecessary reloads
Posted by Aaron Coburn <ac...@amherst.edu>.
Your suggestions all sound good to me.
Aaron C.
On Oct 18, 2012, at 12:53 PM, Aaron Peeler <aa...@ncsu.edu> wrote:
> I agree, having the human readable imagname is important from a system
> admin view and I'd also like to keep that.
>
> So if ya'll are good with the suggestions as it relates to retrieving
> the currentimage.txt contents, I'll proceed. Then we can start another
> issue that addresses setting a shorter naming scheme for newly created
> images that would accommodate vcenter, openstack, or the next gen
> provisioning module.
>
> -Aaron P.
>
> On Thu, Oct 18, 2012 at 12:31 PM, Josh Thompson <jo...@ncsu.edu> wrote:
>> Aaron P., Andy, and I have typically had the same thoughts.
>>
>> Josh
>>
>> On Thursday, October 18, 2012 3:38:31 PM Aaron Coburn wrote:
>>> I would also really encourage us to keep the imagenames somewhat human
>>> readable.
>>>
>>> Otherwise, if someone needs to make any manual changes to the virtual disks
>>> stored on a VM Host (i.e. to remove outdated virtual disks from a
>>> datastore), opaque names would only result in greater troubles.
>>>
>>> Aaron Coburn
>>>
>>> On Oct 18, 2012, at 11:27 AM, Aaron Peeler <aa...@ncsu.edu>
>>>
>>> wrote:
>>>> Hi Mark,
>>>>
>>>> Yes, in trying to make the imagename human readable it is keeping
>>>> state in the imagename on what type of image it is(vmware, partimage,
>>>> Kickstart, lab, etc), which has made it problematic when adding new
>>>> provisioning systems that require shorter names.
>>>>
>>>> There should be another jira issue/discussion on how the imagenames
>>>> are formatted on image creation. Maybe for the 2.4 release.
>>>>
>>>> The suggestion is to update the code that checks what is loaded to use
>>>> the imagerevision_id which is very short will make the format of the
>>>> internal imagenames less important from VCL's perspective. This will
>>>> allow for the name of the raw files stored in the image_library can be
>>>> more flexible, (as long as they match what is in the database).
>>>>
>>>> For this, the suggested changes would not make any changes to the
>>>> contents of currentimage.txt or the stored imagename, but only in how
>>>> it is processed in the core OS routines and with the optional
>>>> variables I suggested one can query for the imagename also.
>>>>
>>>> i.e.
>>>> The default return value of get_current_image_name
>>>> $self->os->get_current_image_name()
>>>> returns imagerevision_id
>>>>
>>>> it would also accept variables to return any other content
>>>> $self->os->get_current_image_name(current_image_name)
>>>> returns current_image_name
>>>> $self->os->get_current_image_name(id)
>>>> returns image id
>>>> $self->os->get_current_image_name(imagerevision_datecreated)
>>>> returns date created
>>>> $self->os->get_current_image_name(prettyname)
>>>> returns prettyname
>>>> $self->os->get_current_image_name(vcld_post_load)
>>>>
>>>>
>>>> Actually on the backend it's not too much to change, just a few
>>>> modules. I would also probably edit routine name:
>>>> from $self->os->get_current_image_name to $self->os->get_current_image
>>>>
>>>>
>>>> Aaron
>>>>
>>>> On Thu, Oct 18, 2012 at 10:57 AM, Mark Gardner <mk...@vt.edu> wrote:
>>>>> Aaron,
>>>>>
>>>>> Is this a symptom of a greater problem where we are encoding state
>>>>> into the image names (thereby causing the names to be rather long)?
>>>>>
>>>>> It seems to me that encoding state into the image name is a little
>>>>> fragile. Perhaps there ought to be a table in the DB that holds the
>>>>> mapping so that the image name no longer has any significance and
>>>>> hence can be shorter (and hence less likely to exceed the silent limit
>>>>> in vSphere).
>>>>>
>>>>> Example: Instead of "vmwarewinxp-base7-v1" we would have a table that
>>>>> might contain: ImageName='96A28E48F2', HypervisorType='vmware',
>>>>> OSType='winxp', label='base7', version=1, etc.
>>>>>
>>>>> The downside is that the image names no longer are "human parseable"
>>>>> and hence we probably should provide a way to query the DB given an
>>>>> opaque image name and get back the human readable data that is
>>>>> currently encode in the image name.
>>>>>
>>>>> Unfortunately this would not be a small change as I believe the use of
>>>>> the encoded image names is scattered through many places. I do believe
>>>>> it would be a worth while change though.
>>>>>
>>>>> Mark
>>>>>
>>>>> On Thu, Oct 18, 2012 at 10:20 AM, Aaron Peeler <fa...@ncsu.edu> wrote:
>>>>>> I can work on this one.
>>>>>>
>>>>>> My thoughts to proceed, first convert OS.pm:sub
>>>>>> get_currentimage_txt_contents to return a hash of the contents of
>>>>>> currentimage.txt then update anything that currently depends on it
>>>>>> which I think is only OS.pm:sub get_current_image_name
>>>>>>
>>>>>> Update get_current_image_name to return optional values, imageid,
>>>>>> imagerevision_id, imagename, etc. The default would be
>>>>>> imagerevision_id.
>>>>>>
>>>>>> Then in update any provisioning modules node_status routine to start
>>>>>> using
>>>>>> $self->os->get_current_image_name()
>>>>>>
>>>>>> based on the desired return value, then node_status or other routines
>>>>>> can then compare what is loaded vs what is requested.
>>>>>>
>>>>>> and lastest remove the old utils::_getcurrentimage routine
>>>>>>
>>>>>> This would allow for changes in the name like shorter and less strict
>>>>>> naming of the internal imagenames.
>>>>>>
>>>>>> Thoughts or comments before I proceed?
>>>>>>
>>>>>> Aaron P.
>>>>>>
>>>>>> On Fri, Sep 21, 2012 at 11:07 AM, Aaron Peeler (JIRA) <ji...@apache.org>
>> wrote:
>>>>>>> [
>>>>>>> https://issues.apache.org/jira/browse/VCL-630?page=com.atlassian.ji
>>>>>>> ra.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13
>>>>>>> 460545#comment-13460545 ]>>>>
>>>>>>> Aaron Peeler commented on VCL-630:
>>>>>>> ----------------------------------
>>>>>>>
>>>>>>> follow-up - there is a get_currentimage_txt_contents in the OS.pm
>>>>>>> module already. I think the best path would be to create a
>>>>>>> OS::get_currentimage_imagerevision_id routine which parses the
>>>>>>> contents and returns the imagerevisionid on the loaded node. Then
>>>>>>> update the provisioning module's node_status routines to use this
>>>>>>> instead.>>>>
>>>>>>>> currentimage.txt name may conflict with imagerevision name, causing
>>>>>>>> unnecessary reloads
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> ------------------>>>>>
>>>>>>>> Key: VCL-630
>>>>>>>> URL: https://issues.apache.org/jira/browse/VCL-630
>>>>>>>>
>>>>>>>> Project: VCL
>>>>>>>>
>>>>>>>> Issue Type: Bug
>>>>>>>> Components: vcld (backend)
>>>>>>>>
>>>>>>>> Affects Versions: 2.3
>>>>>>>>
>>>>>>>> Environment: accessing the first revision of an image on a
>>>>>>>> clustered (vCenter) VMware host. This only affects images
>>>>>>>> whose name (cleaned of non-alphanumeric characters) exceeds 29
>>>>>>>> characters.>>>>>
>>>>>>>> Reporter: Aaron Coburn
>>>>>>>> Assignee: Aaron Coburn
>>>>>>>>
>>>>>>>> Fix For: 2.4, 2.3.1
>>>>>>>>
>>>>>>>> A few words to provide some background: when the VCL uses a clustered
>>>>>>>> (vCenter) VMware host, one cannot simply copy virtual disks as part
>>>>>>>> of the image capture process; instead, the VM is cloned. This,
>>>>>>>> however, exposed an apparently undocumented feature of VMware in that
>>>>>>>> virtual disk names exceeding 29 characters are silently truncated.
>>>>>>>> This creates significant problems for the VCL. The current solution
>>>>>>>> (for 2.3) is to update the image name in the database after
>>>>>>>> successfully capturing an image. This way, the VCL knows where to
>>>>>>>> find the virtual disk in the VMware datastore. The problem with this
>>>>>>>> approach is that this change isn't registered with the system until
>>>>>>>> after the image has been captured. At this point, however, the
>>>>>>>> currentimage.txt file has already been written to the disk and if the
>>>>>>>> name changes ex post facto, then the problem simply re-emerges in a
>>>>>>>> different form when a user makes a reservation for the image.
>>>>>>>> Assuming the VM is available and otherwise ready to grant an
>>>>>>>> immediate reservation, the reclaim.pm:process() subroutine checks the
>>>>>>>> VM name listed in the currentimage.txt file and finds a conflict in
>>>>>>>> the name listed on the OS and the name in the database. An immediate
>>>>>>>> solution for users is to simply make a new revision of the same
>>>>>>>> image. That second revision will have a shortened name in both
>>>>>>>> currentimage.txt and the imagerevision table. There are two ways that
>>>>>>>> this could be correctly solved in the perl code: either by updating
>>>>>>>> the image name earlier in the provisioning module's capture()
>>>>>>>> subroutine or by extracting other information from currentimage.txt,
>>>>>>>> such as imagerevision_id. While the first approach would be easier to
>>>>>>>> implement, the second approach seems to be more architecturally
>>>>>>>> sound. That is, rather than relying on a comparison of two strings,
>>>>>>>> the test would be between two integers, one of which is the primary
>>>>>>>> key of the database table in question. Any thoughts on the best way
>>>>>>>> to proceed?
>>>>>>>
>>>>>>> --
>>>>>>> This message is automatically generated by JIRA.
>>>>>>> If you think it was sent incorrectly, please contact your JIRA
>>>>>>> administrators For more information on JIRA, see:
>>>>>>> http://www.atlassian.com/software/jira>>>
>>>>>> --
>>>>>> Aaron Peeler
>>>>>> Program Manager
>>>>>> Virtual Computing Lab
>>>>>> NC State University
>>>>>>
>>>>>> All electronic mail messages in connection with State business which
>>>>>> are sent to or received by this account are subject to the NC Public
>>>>>> Records Law and may be disclosed to third parties.
>>>>>
>>>>> --
>>>>> Mark Gardner
>>>>> --
>> --
>> -------------------------------
>> Josh Thompson
>> Systems Programmer
>> Advanced Computing | VCL Developer
>> North Carolina State University
>>
>> Josh_Thompson@ncsu.edu
>> 919-515-5323
>>
>> my GPG/PGP key can be found at pgp.mit.edu
>>
>> All electronic mail messages in connection with State business which
>> are sent to or received by this account are subject to the NC Public
>> Records Law and may be disclosed to third parties.
>
>
>
> --
> Aaron Peeler
> Program Manager
> Virtual Computing Lab
> NC State University
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
Re: [jira] [Commented] (VCL-630) currentimage.txt name may conflict
with imagerevision name, causing unnecessary reloads
Posted by Aaron Peeler <aa...@ncsu.edu>.
I agree, having the human readable imagname is important from a system
admin view and I'd also like to keep that.
So if ya'll are good with the suggestions as it relates to retrieving
the currentimage.txt contents, I'll proceed. Then we can start another
issue that addresses setting a shorter naming scheme for newly created
images that would accommodate vcenter, openstack, or the next gen
provisioning module.
-Aaron P.
On Thu, Oct 18, 2012 at 12:31 PM, Josh Thompson <jo...@ncsu.edu> wrote:
> Aaron P., Andy, and I have typically had the same thoughts.
>
> Josh
>
> On Thursday, October 18, 2012 3:38:31 PM Aaron Coburn wrote:
>> I would also really encourage us to keep the imagenames somewhat human
>> readable.
>>
>> Otherwise, if someone needs to make any manual changes to the virtual disks
>> stored on a VM Host (i.e. to remove outdated virtual disks from a
>> datastore), opaque names would only result in greater troubles.
>>
>> Aaron Coburn
>>
>> On Oct 18, 2012, at 11:27 AM, Aaron Peeler <aa...@ncsu.edu>
>>
>> wrote:
>> > Hi Mark,
>> >
>> > Yes, in trying to make the imagename human readable it is keeping
>> > state in the imagename on what type of image it is(vmware, partimage,
>> > Kickstart, lab, etc), which has made it problematic when adding new
>> > provisioning systems that require shorter names.
>> >
>> > There should be another jira issue/discussion on how the imagenames
>> > are formatted on image creation. Maybe for the 2.4 release.
>> >
>> > The suggestion is to update the code that checks what is loaded to use
>> > the imagerevision_id which is very short will make the format of the
>> > internal imagenames less important from VCL's perspective. This will
>> > allow for the name of the raw files stored in the image_library can be
>> > more flexible, (as long as they match what is in the database).
>> >
>> > For this, the suggested changes would not make any changes to the
>> > contents of currentimage.txt or the stored imagename, but only in how
>> > it is processed in the core OS routines and with the optional
>> > variables I suggested one can query for the imagename also.
>> >
>> > i.e.
>> > The default return value of get_current_image_name
>> > $self->os->get_current_image_name()
>> > returns imagerevision_id
>> >
>> > it would also accept variables to return any other content
>> > $self->os->get_current_image_name(current_image_name)
>> > returns current_image_name
>> > $self->os->get_current_image_name(id)
>> > returns image id
>> > $self->os->get_current_image_name(imagerevision_datecreated)
>> > returns date created
>> > $self->os->get_current_image_name(prettyname)
>> > returns prettyname
>> > $self->os->get_current_image_name(vcld_post_load)
>> >
>> >
>> > Actually on the backend it's not too much to change, just a few
>> > modules. I would also probably edit routine name:
>> > from $self->os->get_current_image_name to $self->os->get_current_image
>> >
>> >
>> > Aaron
>> >
>> > On Thu, Oct 18, 2012 at 10:57 AM, Mark Gardner <mk...@vt.edu> wrote:
>> >> Aaron,
>> >>
>> >> Is this a symptom of a greater problem where we are encoding state
>> >> into the image names (thereby causing the names to be rather long)?
>> >>
>> >> It seems to me that encoding state into the image name is a little
>> >> fragile. Perhaps there ought to be a table in the DB that holds the
>> >> mapping so that the image name no longer has any significance and
>> >> hence can be shorter (and hence less likely to exceed the silent limit
>> >> in vSphere).
>> >>
>> >> Example: Instead of "vmwarewinxp-base7-v1" we would have a table that
>> >> might contain: ImageName='96A28E48F2', HypervisorType='vmware',
>> >> OSType='winxp', label='base7', version=1, etc.
>> >>
>> >> The downside is that the image names no longer are "human parseable"
>> >> and hence we probably should provide a way to query the DB given an
>> >> opaque image name and get back the human readable data that is
>> >> currently encode in the image name.
>> >>
>> >> Unfortunately this would not be a small change as I believe the use of
>> >> the encoded image names is scattered through many places. I do believe
>> >> it would be a worth while change though.
>> >>
>> >> Mark
>> >>
>> >> On Thu, Oct 18, 2012 at 10:20 AM, Aaron Peeler <fa...@ncsu.edu> wrote:
>> >>> I can work on this one.
>> >>>
>> >>> My thoughts to proceed, first convert OS.pm:sub
>> >>> get_currentimage_txt_contents to return a hash of the contents of
>> >>> currentimage.txt then update anything that currently depends on it
>> >>> which I think is only OS.pm:sub get_current_image_name
>> >>>
>> >>> Update get_current_image_name to return optional values, imageid,
>> >>> imagerevision_id, imagename, etc. The default would be
>> >>> imagerevision_id.
>> >>>
>> >>> Then in update any provisioning modules node_status routine to start
>> >>> using
>> >>> $self->os->get_current_image_name()
>> >>>
>> >>> based on the desired return value, then node_status or other routines
>> >>> can then compare what is loaded vs what is requested.
>> >>>
>> >>> and lastest remove the old utils::_getcurrentimage routine
>> >>>
>> >>> This would allow for changes in the name like shorter and less strict
>> >>> naming of the internal imagenames.
>> >>>
>> >>> Thoughts or comments before I proceed?
>> >>>
>> >>> Aaron P.
>> >>>
>> >>> On Fri, Sep 21, 2012 at 11:07 AM, Aaron Peeler (JIRA) <ji...@apache.org>
> wrote:
>> >>>> [
>> >>>> https://issues.apache.org/jira/browse/VCL-630?page=com.atlassian.ji
>> >>>> ra.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13
>> >>>> 460545#comment-13460545 ]>>>>
>> >>>> Aaron Peeler commented on VCL-630:
>> >>>> ----------------------------------
>> >>>>
>> >>>> follow-up - there is a get_currentimage_txt_contents in the OS.pm
>> >>>> module already. I think the best path would be to create a
>> >>>> OS::get_currentimage_imagerevision_id routine which parses the
>> >>>> contents and returns the imagerevisionid on the loaded node. Then
>> >>>> update the provisioning module's node_status routines to use this
>> >>>> instead.>>>>
>> >>>>> currentimage.txt name may conflict with imagerevision name, causing
>> >>>>> unnecessary reloads
>> >>>>> ---------------------------------------------------------------------
>> >>>>> ------------------>>>>>
>> >>>>> Key: VCL-630
>> >>>>> URL: https://issues.apache.org/jira/browse/VCL-630
>> >>>>>
>> >>>>> Project: VCL
>> >>>>>
>> >>>>> Issue Type: Bug
>> >>>>> Components: vcld (backend)
>> >>>>>
>> >>>>> Affects Versions: 2.3
>> >>>>>
>> >>>>> Environment: accessing the first revision of an image on a
>> >>>>> clustered (vCenter) VMware host. This only affects images
>> >>>>> whose name (cleaned of non-alphanumeric characters) exceeds 29
>> >>>>> characters.>>>>>
>> >>>>> Reporter: Aaron Coburn
>> >>>>> Assignee: Aaron Coburn
>> >>>>>
>> >>>>> Fix For: 2.4, 2.3.1
>> >>>>>
>> >>>>> A few words to provide some background: when the VCL uses a clustered
>> >>>>> (vCenter) VMware host, one cannot simply copy virtual disks as part
>> >>>>> of the image capture process; instead, the VM is cloned. This,
>> >>>>> however, exposed an apparently undocumented feature of VMware in that
>> >>>>> virtual disk names exceeding 29 characters are silently truncated.
>> >>>>> This creates significant problems for the VCL. The current solution
>> >>>>> (for 2.3) is to update the image name in the database after
>> >>>>> successfully capturing an image. This way, the VCL knows where to
>> >>>>> find the virtual disk in the VMware datastore. The problem with this
>> >>>>> approach is that this change isn't registered with the system until
>> >>>>> after the image has been captured. At this point, however, the
>> >>>>> currentimage.txt file has already been written to the disk and if the
>> >>>>> name changes ex post facto, then the problem simply re-emerges in a
>> >>>>> different form when a user makes a reservation for the image.
>> >>>>> Assuming the VM is available and otherwise ready to grant an
>> >>>>> immediate reservation, the reclaim.pm:process() subroutine checks the
>> >>>>> VM name listed in the currentimage.txt file and finds a conflict in
>> >>>>> the name listed on the OS and the name in the database. An immediate
>> >>>>> solution for users is to simply make a new revision of the same
>> >>>>> image. That second revision will have a shortened name in both
>> >>>>> currentimage.txt and the imagerevision table. There are two ways that
>> >>>>> this could be correctly solved in the perl code: either by updating
>> >>>>> the image name earlier in the provisioning module's capture()
>> >>>>> subroutine or by extracting other information from currentimage.txt,
>> >>>>> such as imagerevision_id. While the first approach would be easier to
>> >>>>> implement, the second approach seems to be more architecturally
>> >>>>> sound. That is, rather than relying on a comparison of two strings,
>> >>>>> the test would be between two integers, one of which is the primary
>> >>>>> key of the database table in question. Any thoughts on the best way
>> >>>>> to proceed?
>> >>>>
>> >>>> --
>> >>>> This message is automatically generated by JIRA.
>> >>>> If you think it was sent incorrectly, please contact your JIRA
>> >>>> administrators For more information on JIRA, see:
>> >>>> http://www.atlassian.com/software/jira>>>
>> >>> --
>> >>> Aaron Peeler
>> >>> Program Manager
>> >>> Virtual Computing Lab
>> >>> NC State University
>> >>>
>> >>> All electronic mail messages in connection with State business which
>> >>> are sent to or received by this account are subject to the NC Public
>> >>> Records Law and may be disclosed to third parties.
>> >>
>> >> --
>> >> Mark Gardner
>> >> --
> --
> -------------------------------
> Josh Thompson
> Systems Programmer
> Advanced Computing | VCL Developer
> North Carolina State University
>
> Josh_Thompson@ncsu.edu
> 919-515-5323
>
> my GPG/PGP key can be found at pgp.mit.edu
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
--
Aaron Peeler
Program Manager
Virtual Computing Lab
NC State University
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.
Re: [jira] [Commented] (VCL-630) currentimage.txt name may conflict with imagerevision name, causing unnecessary reloads
Posted by Josh Thompson <jo...@ncsu.edu>.
Aaron P., Andy, and I have typically had the same thoughts.
Josh
On Thursday, October 18, 2012 3:38:31 PM Aaron Coburn wrote:
> I would also really encourage us to keep the imagenames somewhat human
> readable.
>
> Otherwise, if someone needs to make any manual changes to the virtual disks
> stored on a VM Host (i.e. to remove outdated virtual disks from a
> datastore), opaque names would only result in greater troubles.
>
> Aaron Coburn
>
> On Oct 18, 2012, at 11:27 AM, Aaron Peeler <aa...@ncsu.edu>
>
> wrote:
> > Hi Mark,
> >
> > Yes, in trying to make the imagename human readable it is keeping
> > state in the imagename on what type of image it is(vmware, partimage,
> > Kickstart, lab, etc), which has made it problematic when adding new
> > provisioning systems that require shorter names.
> >
> > There should be another jira issue/discussion on how the imagenames
> > are formatted on image creation. Maybe for the 2.4 release.
> >
> > The suggestion is to update the code that checks what is loaded to use
> > the imagerevision_id which is very short will make the format of the
> > internal imagenames less important from VCL's perspective. This will
> > allow for the name of the raw files stored in the image_library can be
> > more flexible, (as long as they match what is in the database).
> >
> > For this, the suggested changes would not make any changes to the
> > contents of currentimage.txt or the stored imagename, but only in how
> > it is processed in the core OS routines and with the optional
> > variables I suggested one can query for the imagename also.
> >
> > i.e.
> > The default return value of get_current_image_name
> > $self->os->get_current_image_name()
> > returns imagerevision_id
> >
> > it would also accept variables to return any other content
> > $self->os->get_current_image_name(current_image_name)
> > returns current_image_name
> > $self->os->get_current_image_name(id)
> > returns image id
> > $self->os->get_current_image_name(imagerevision_datecreated)
> > returns date created
> > $self->os->get_current_image_name(prettyname)
> > returns prettyname
> > $self->os->get_current_image_name(vcld_post_load)
> >
> >
> > Actually on the backend it's not too much to change, just a few
> > modules. I would also probably edit routine name:
> > from $self->os->get_current_image_name to $self->os->get_current_image
> >
> >
> > Aaron
> >
> > On Thu, Oct 18, 2012 at 10:57 AM, Mark Gardner <mk...@vt.edu> wrote:
> >> Aaron,
> >>
> >> Is this a symptom of a greater problem where we are encoding state
> >> into the image names (thereby causing the names to be rather long)?
> >>
> >> It seems to me that encoding state into the image name is a little
> >> fragile. Perhaps there ought to be a table in the DB that holds the
> >> mapping so that the image name no longer has any significance and
> >> hence can be shorter (and hence less likely to exceed the silent limit
> >> in vSphere).
> >>
> >> Example: Instead of "vmwarewinxp-base7-v1" we would have a table that
> >> might contain: ImageName='96A28E48F2', HypervisorType='vmware',
> >> OSType='winxp', label='base7', version=1, etc.
> >>
> >> The downside is that the image names no longer are "human parseable"
> >> and hence we probably should provide a way to query the DB given an
> >> opaque image name and get back the human readable data that is
> >> currently encode in the image name.
> >>
> >> Unfortunately this would not be a small change as I believe the use of
> >> the encoded image names is scattered through many places. I do believe
> >> it would be a worth while change though.
> >>
> >> Mark
> >>
> >> On Thu, Oct 18, 2012 at 10:20 AM, Aaron Peeler <fa...@ncsu.edu> wrote:
> >>> I can work on this one.
> >>>
> >>> My thoughts to proceed, first convert OS.pm:sub
> >>> get_currentimage_txt_contents to return a hash of the contents of
> >>> currentimage.txt then update anything that currently depends on it
> >>> which I think is only OS.pm:sub get_current_image_name
> >>>
> >>> Update get_current_image_name to return optional values, imageid,
> >>> imagerevision_id, imagename, etc. The default would be
> >>> imagerevision_id.
> >>>
> >>> Then in update any provisioning modules node_status routine to start
> >>> using
> >>> $self->os->get_current_image_name()
> >>>
> >>> based on the desired return value, then node_status or other routines
> >>> can then compare what is loaded vs what is requested.
> >>>
> >>> and lastest remove the old utils::_getcurrentimage routine
> >>>
> >>> This would allow for changes in the name like shorter and less strict
> >>> naming of the internal imagenames.
> >>>
> >>> Thoughts or comments before I proceed?
> >>>
> >>> Aaron P.
> >>>
> >>> On Fri, Sep 21, 2012 at 11:07 AM, Aaron Peeler (JIRA) <ji...@apache.org>
wrote:
> >>>> [
> >>>> https://issues.apache.org/jira/browse/VCL-630?page=com.atlassian.ji
> >>>> ra.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13
> >>>> 460545#comment-13460545 ]>>>>
> >>>> Aaron Peeler commented on VCL-630:
> >>>> ----------------------------------
> >>>>
> >>>> follow-up - there is a get_currentimage_txt_contents in the OS.pm
> >>>> module already. I think the best path would be to create a
> >>>> OS::get_currentimage_imagerevision_id routine which parses the
> >>>> contents and returns the imagerevisionid on the loaded node. Then
> >>>> update the provisioning module's node_status routines to use this
> >>>> instead.>>>>
> >>>>> currentimage.txt name may conflict with imagerevision name, causing
> >>>>> unnecessary reloads
> >>>>> ---------------------------------------------------------------------
> >>>>> ------------------>>>>>
> >>>>> Key: VCL-630
> >>>>> URL: https://issues.apache.org/jira/browse/VCL-630
> >>>>>
> >>>>> Project: VCL
> >>>>>
> >>>>> Issue Type: Bug
> >>>>> Components: vcld (backend)
> >>>>>
> >>>>> Affects Versions: 2.3
> >>>>>
> >>>>> Environment: accessing the first revision of an image on a
> >>>>> clustered (vCenter) VMware host. This only affects images
> >>>>> whose name (cleaned of non-alphanumeric characters) exceeds 29
> >>>>> characters.>>>>>
> >>>>> Reporter: Aaron Coburn
> >>>>> Assignee: Aaron Coburn
> >>>>>
> >>>>> Fix For: 2.4, 2.3.1
> >>>>>
> >>>>> A few words to provide some background: when the VCL uses a clustered
> >>>>> (vCenter) VMware host, one cannot simply copy virtual disks as part
> >>>>> of the image capture process; instead, the VM is cloned. This,
> >>>>> however, exposed an apparently undocumented feature of VMware in that
> >>>>> virtual disk names exceeding 29 characters are silently truncated.
> >>>>> This creates significant problems for the VCL. The current solution
> >>>>> (for 2.3) is to update the image name in the database after
> >>>>> successfully capturing an image. This way, the VCL knows where to
> >>>>> find the virtual disk in the VMware datastore. The problem with this
> >>>>> approach is that this change isn't registered with the system until
> >>>>> after the image has been captured. At this point, however, the
> >>>>> currentimage.txt file has already been written to the disk and if the
> >>>>> name changes ex post facto, then the problem simply re-emerges in a
> >>>>> different form when a user makes a reservation for the image.
> >>>>> Assuming the VM is available and otherwise ready to grant an
> >>>>> immediate reservation, the reclaim.pm:process() subroutine checks the
> >>>>> VM name listed in the currentimage.txt file and finds a conflict in
> >>>>> the name listed on the OS and the name in the database. An immediate
> >>>>> solution for users is to simply make a new revision of the same
> >>>>> image. That second revision will have a shortened name in both
> >>>>> currentimage.txt and the imagerevision table. There are two ways that
> >>>>> this could be correctly solved in the perl code: either by updating
> >>>>> the image name earlier in the provisioning module's capture()
> >>>>> subroutine or by extracting other information from currentimage.txt,
> >>>>> such as imagerevision_id. While the first approach would be easier to
> >>>>> implement, the second approach seems to be more architecturally
> >>>>> sound. That is, rather than relying on a comparison of two strings,
> >>>>> the test would be between two integers, one of which is the primary
> >>>>> key of the database table in question. Any thoughts on the best way
> >>>>> to proceed?
> >>>>
> >>>> --
> >>>> This message is automatically generated by JIRA.
> >>>> If you think it was sent incorrectly, please contact your JIRA
> >>>> administrators For more information on JIRA, see:
> >>>> http://www.atlassian.com/software/jira>>>
> >>> --
> >>> Aaron Peeler
> >>> Program Manager
> >>> Virtual Computing Lab
> >>> NC State University
> >>>
> >>> All electronic mail messages in connection with State business which
> >>> are sent to or received by this account are subject to the NC Public
> >>> Records Law and may be disclosed to third parties.
> >>
> >> --
> >> Mark Gardner
> >> --
--
-------------------------------
Josh Thompson
Systems Programmer
Advanced Computing | VCL Developer
North Carolina State University
Josh_Thompson@ncsu.edu
919-515-5323
my GPG/PGP key can be found at pgp.mit.edu
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.
Re: [jira] [Commented] (VCL-630) currentimage.txt name may conflict
with imagerevision name, causing unnecessary reloads
Posted by Aaron Coburn <ac...@amherst.edu>.
I would also really encourage us to keep the imagenames somewhat human readable.
Otherwise, if someone needs to make any manual changes to the virtual disks stored on a VM Host (i.e. to remove outdated virtual disks from a datastore), opaque names would only result in greater troubles.
Aaron Coburn
On Oct 18, 2012, at 11:27 AM, Aaron Peeler <aa...@ncsu.edu>
wrote:
> Hi Mark,
>
> Yes, in trying to make the imagename human readable it is keeping
> state in the imagename on what type of image it is(vmware, partimage,
> Kickstart, lab, etc), which has made it problematic when adding new
> provisioning systems that require shorter names.
>
> There should be another jira issue/discussion on how the imagenames
> are formatted on image creation. Maybe for the 2.4 release.
>
> The suggestion is to update the code that checks what is loaded to use
> the imagerevision_id which is very short will make the format of the
> internal imagenames less important from VCL's perspective. This will
> allow for the name of the raw files stored in the image_library can be
> more flexible, (as long as they match what is in the database).
>
> For this, the suggested changes would not make any changes to the
> contents of currentimage.txt or the stored imagename, but only in how
> it is processed in the core OS routines and with the optional
> variables I suggested one can query for the imagename also.
>
> i.e.
> The default return value of get_current_image_name
> $self->os->get_current_image_name()
> returns imagerevision_id
>
> it would also accept variables to return any other content
> $self->os->get_current_image_name(current_image_name)
> returns current_image_name
> $self->os->get_current_image_name(id)
> returns image id
> $self->os->get_current_image_name(imagerevision_datecreated)
> returns date created
> $self->os->get_current_image_name(prettyname)
> returns prettyname
> $self->os->get_current_image_name(vcld_post_load)
>
>
> Actually on the backend it's not too much to change, just a few
> modules. I would also probably edit routine name:
> from $self->os->get_current_image_name to $self->os->get_current_image
>
>
> Aaron
>
> On Thu, Oct 18, 2012 at 10:57 AM, Mark Gardner <mk...@vt.edu> wrote:
>> Aaron,
>>
>> Is this a symptom of a greater problem where we are encoding state
>> into the image names (thereby causing the names to be rather long)?
>>
>> It seems to me that encoding state into the image name is a little
>> fragile. Perhaps there ought to be a table in the DB that holds the
>> mapping so that the image name no longer has any significance and
>> hence can be shorter (and hence less likely to exceed the silent limit
>> in vSphere).
>>
>> Example: Instead of "vmwarewinxp-base7-v1" we would have a table that
>> might contain: ImageName='96A28E48F2', HypervisorType='vmware',
>> OSType='winxp', label='base7', version=1, etc.
>>
>> The downside is that the image names no longer are "human parseable"
>> and hence we probably should provide a way to query the DB given an
>> opaque image name and get back the human readable data that is
>> currently encode in the image name.
>>
>> Unfortunately this would not be a small change as I believe the use of
>> the encoded image names is scattered through many places. I do believe
>> it would be a worth while change though.
>>
>> Mark
>>
>> On Thu, Oct 18, 2012 at 10:20 AM, Aaron Peeler <fa...@ncsu.edu> wrote:
>>> I can work on this one.
>>>
>>> My thoughts to proceed, first convert OS.pm:sub
>>> get_currentimage_txt_contents to return a hash of the contents of
>>> currentimage.txt then update anything that currently depends on it
>>> which I think is only OS.pm:sub get_current_image_name
>>>
>>> Update get_current_image_name to return optional values, imageid,
>>> imagerevision_id, imagename, etc. The default would be
>>> imagerevision_id.
>>>
>>> Then in update any provisioning modules node_status routine to start using
>>> $self->os->get_current_image_name()
>>>
>>> based on the desired return value, then node_status or other routines
>>> can then compare what is loaded vs what is requested.
>>>
>>> and lastest remove the old utils::_getcurrentimage routine
>>>
>>> This would allow for changes in the name like shorter and less strict
>>> naming of the internal imagenames.
>>>
>>> Thoughts or comments before I proceed?
>>>
>>> Aaron P.
>>>
>>>
>>> On Fri, Sep 21, 2012 at 11:07 AM, Aaron Peeler (JIRA) <ji...@apache.org> wrote:
>>>>
>>>> [ https://issues.apache.org/jira/browse/VCL-630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460545#comment-13460545 ]
>>>>
>>>> Aaron Peeler commented on VCL-630:
>>>> ----------------------------------
>>>>
>>>> follow-up - there is a get_currentimage_txt_contents in the OS.pm module already. I think the best path would be to create a OS::get_currentimage_imagerevision_id routine which parses the contents and returns the imagerevisionid on the loaded node. Then update the provisioning module's node_status routines to use this instead.
>>>>
>>>>> currentimage.txt name may conflict with imagerevision name, causing unnecessary reloads
>>>>> ---------------------------------------------------------------------------------------
>>>>>
>>>>> Key: VCL-630
>>>>> URL: https://issues.apache.org/jira/browse/VCL-630
>>>>> Project: VCL
>>>>> Issue Type: Bug
>>>>> Components: vcld (backend)
>>>>> Affects Versions: 2.3
>>>>> Environment: accessing the first revision of an image on a clustered (vCenter) VMware host. This only affects images whose name (cleaned of non-alphanumeric characters) exceeds 29 characters.
>>>>> Reporter: Aaron Coburn
>>>>> Assignee: Aaron Coburn
>>>>> Fix For: 2.4, 2.3.1
>>>>>
>>>>>
>>>>> A few words to provide some background: when the VCL uses a clustered (vCenter) VMware host, one cannot simply copy virtual disks as part of the image capture process; instead, the VM is cloned. This, however, exposed an apparently undocumented feature of VMware in that virtual disk names exceeding 29 characters are silently truncated. This creates significant problems for the VCL. The current solution (for 2.3) is to update the image name in the database after successfully capturing an image. This way, the VCL knows where to find the virtual disk in the VMware datastore.
>>>>> The problem with this approach is that this change isn't registered with the system until after the image has been captured. At this point, however, the currentimage.txt file has already been written to the disk and if the name changes ex post facto, then the problem simply re-emerges in a different form when a user makes a reservation for the image.
>>>>> Assuming the VM is available and otherwise ready to grant an immediate reservation, the reclaim.pm:process() subroutine checks the VM name listed in the currentimage.txt file and finds a conflict in the name listed on the OS and the name in the database.
>>>>> An immediate solution for users is to simply make a new revision of the same image. That second revision will have a shortened name in both currentimage.txt and the imagerevision table.
>>>>> There are two ways that this could be correctly solved in the perl code: either by updating the image name earlier in the provisioning module's capture() subroutine or by extracting other information from currentimage.txt, such as imagerevision_id.
>>>>> While the first approach would be easier to implement, the second approach seems to be more architecturally sound. That is, rather than relying on a comparison of two strings, the test would be between two integers, one of which is the primary key of the database table in question.
>>>>> Any thoughts on the best way to proceed?
>>>>
>>>> --
>>>> This message is automatically generated by JIRA.
>>>> If you think it was sent incorrectly, please contact your JIRA administrators
>>>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>>
>>>
>>>
>>> --
>>> Aaron Peeler
>>> Program Manager
>>> Virtual Computing Lab
>>> NC State University
>>>
>>> All electronic mail messages in connection with State business which
>>> are sent to or received by this account are subject to the NC Public
>>> Records Law and may be disclosed to third parties.
>>
>>
>>
>> --
>> Mark Gardner
>> --
>
>
>
> --
> Aaron Peeler
> Program Manager
> Virtual Computing Lab
> NC State University
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
Re: [jira] [Commented] (VCL-630) currentimage.txt name may conflict
with imagerevision name, causing unnecessary reloads
Posted by Aaron Peeler <aa...@ncsu.edu>.
Hi Mark,
Yes, in trying to make the imagename human readable it is keeping
state in the imagename on what type of image it is(vmware, partimage,
Kickstart, lab, etc), which has made it problematic when adding new
provisioning systems that require shorter names.
There should be another jira issue/discussion on how the imagenames
are formatted on image creation. Maybe for the 2.4 release.
The suggestion is to update the code that checks what is loaded to use
the imagerevision_id which is very short will make the format of the
internal imagenames less important from VCL's perspective. This will
allow for the name of the raw files stored in the image_library can be
more flexible, (as long as they match what is in the database).
For this, the suggested changes would not make any changes to the
contents of currentimage.txt or the stored imagename, but only in how
it is processed in the core OS routines and with the optional
variables I suggested one can query for the imagename also.
i.e.
The default return value of get_current_image_name
$self->os->get_current_image_name()
returns imagerevision_id
it would also accept variables to return any other content
$self->os->get_current_image_name(current_image_name)
returns current_image_name
$self->os->get_current_image_name(id)
returns image id
$self->os->get_current_image_name(imagerevision_datecreated)
returns date created
$self->os->get_current_image_name(prettyname)
returns prettyname
$self->os->get_current_image_name(vcld_post_load)
Actually on the backend it's not too much to change, just a few
modules. I would also probably edit routine name:
from $self->os->get_current_image_name to $self->os->get_current_image
Aaron
On Thu, Oct 18, 2012 at 10:57 AM, Mark Gardner <mk...@vt.edu> wrote:
> Aaron,
>
> Is this a symptom of a greater problem where we are encoding state
> into the image names (thereby causing the names to be rather long)?
>
> It seems to me that encoding state into the image name is a little
> fragile. Perhaps there ought to be a table in the DB that holds the
> mapping so that the image name no longer has any significance and
> hence can be shorter (and hence less likely to exceed the silent limit
> in vSphere).
>
> Example: Instead of "vmwarewinxp-base7-v1" we would have a table that
> might contain: ImageName='96A28E48F2', HypervisorType='vmware',
> OSType='winxp', label='base7', version=1, etc.
>
> The downside is that the image names no longer are "human parseable"
> and hence we probably should provide a way to query the DB given an
> opaque image name and get back the human readable data that is
> currently encode in the image name.
>
> Unfortunately this would not be a small change as I believe the use of
> the encoded image names is scattered through many places. I do believe
> it would be a worth while change though.
>
> Mark
>
> On Thu, Oct 18, 2012 at 10:20 AM, Aaron Peeler <fa...@ncsu.edu> wrote:
>> I can work on this one.
>>
>> My thoughts to proceed, first convert OS.pm:sub
>> get_currentimage_txt_contents to return a hash of the contents of
>> currentimage.txt then update anything that currently depends on it
>> which I think is only OS.pm:sub get_current_image_name
>>
>> Update get_current_image_name to return optional values, imageid,
>> imagerevision_id, imagename, etc. The default would be
>> imagerevision_id.
>>
>> Then in update any provisioning modules node_status routine to start using
>> $self->os->get_current_image_name()
>>
>> based on the desired return value, then node_status or other routines
>> can then compare what is loaded vs what is requested.
>>
>> and lastest remove the old utils::_getcurrentimage routine
>>
>> This would allow for changes in the name like shorter and less strict
>> naming of the internal imagenames.
>>
>> Thoughts or comments before I proceed?
>>
>> Aaron P.
>>
>>
>> On Fri, Sep 21, 2012 at 11:07 AM, Aaron Peeler (JIRA) <ji...@apache.org> wrote:
>>>
>>> [ https://issues.apache.org/jira/browse/VCL-630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460545#comment-13460545 ]
>>>
>>> Aaron Peeler commented on VCL-630:
>>> ----------------------------------
>>>
>>> follow-up - there is a get_currentimage_txt_contents in the OS.pm module already. I think the best path would be to create a OS::get_currentimage_imagerevision_id routine which parses the contents and returns the imagerevisionid on the loaded node. Then update the provisioning module's node_status routines to use this instead.
>>>
>>>> currentimage.txt name may conflict with imagerevision name, causing unnecessary reloads
>>>> ---------------------------------------------------------------------------------------
>>>>
>>>> Key: VCL-630
>>>> URL: https://issues.apache.org/jira/browse/VCL-630
>>>> Project: VCL
>>>> Issue Type: Bug
>>>> Components: vcld (backend)
>>>> Affects Versions: 2.3
>>>> Environment: accessing the first revision of an image on a clustered (vCenter) VMware host. This only affects images whose name (cleaned of non-alphanumeric characters) exceeds 29 characters.
>>>> Reporter: Aaron Coburn
>>>> Assignee: Aaron Coburn
>>>> Fix For: 2.4, 2.3.1
>>>>
>>>>
>>>> A few words to provide some background: when the VCL uses a clustered (vCenter) VMware host, one cannot simply copy virtual disks as part of the image capture process; instead, the VM is cloned. This, however, exposed an apparently undocumented feature of VMware in that virtual disk names exceeding 29 characters are silently truncated. This creates significant problems for the VCL. The current solution (for 2.3) is to update the image name in the database after successfully capturing an image. This way, the VCL knows where to find the virtual disk in the VMware datastore.
>>>> The problem with this approach is that this change isn't registered with the system until after the image has been captured. At this point, however, the currentimage.txt file has already been written to the disk and if the name changes ex post facto, then the problem simply re-emerges in a different form when a user makes a reservation for the image.
>>>> Assuming the VM is available and otherwise ready to grant an immediate reservation, the reclaim.pm:process() subroutine checks the VM name listed in the currentimage.txt file and finds a conflict in the name listed on the OS and the name in the database.
>>>> An immediate solution for users is to simply make a new revision of the same image. That second revision will have a shortened name in both currentimage.txt and the imagerevision table.
>>>> There are two ways that this could be correctly solved in the perl code: either by updating the image name earlier in the provisioning module's capture() subroutine or by extracting other information from currentimage.txt, such as imagerevision_id.
>>>> While the first approach would be easier to implement, the second approach seems to be more architecturally sound. That is, rather than relying on a comparison of two strings, the test would be between two integers, one of which is the primary key of the database table in question.
>>>> Any thoughts on the best way to proceed?
>>>
>>> --
>>> This message is automatically generated by JIRA.
>>> If you think it was sent incorrectly, please contact your JIRA administrators
>>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>
>>
>>
>> --
>> Aaron Peeler
>> Program Manager
>> Virtual Computing Lab
>> NC State University
>>
>> All electronic mail messages in connection with State business which
>> are sent to or received by this account are subject to the NC Public
>> Records Law and may be disclosed to third parties.
>
>
>
> --
> Mark Gardner
> --
--
Aaron Peeler
Program Manager
Virtual Computing Lab
NC State University
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.
Re: [jira] [Commented] (VCL-630) currentimage.txt name may conflict
with imagerevision name, causing unnecessary reloads
Posted by Mark Gardner <mk...@vt.edu>.
Aaron,
Is this a symptom of a greater problem where we are encoding state
into the image names (thereby causing the names to be rather long)?
It seems to me that encoding state into the image name is a little
fragile. Perhaps there ought to be a table in the DB that holds the
mapping so that the image name no longer has any significance and
hence can be shorter (and hence less likely to exceed the silent limit
in vSphere).
Example: Instead of "vmwarewinxp-base7-v1" we would have a table that
might contain: ImageName='96A28E48F2', HypervisorType='vmware',
OSType='winxp', label='base7', version=1, etc.
The downside is that the image names no longer are "human parseable"
and hence we probably should provide a way to query the DB given an
opaque image name and get back the human readable data that is
currently encode in the image name.
Unfortunately this would not be a small change as I believe the use of
the encoded image names is scattered through many places. I do believe
it would be a worth while change though.
Mark
On Thu, Oct 18, 2012 at 10:20 AM, Aaron Peeler <fa...@ncsu.edu> wrote:
> I can work on this one.
>
> My thoughts to proceed, first convert OS.pm:sub
> get_currentimage_txt_contents to return a hash of the contents of
> currentimage.txt then update anything that currently depends on it
> which I think is only OS.pm:sub get_current_image_name
>
> Update get_current_image_name to return optional values, imageid,
> imagerevision_id, imagename, etc. The default would be
> imagerevision_id.
>
> Then in update any provisioning modules node_status routine to start using
> $self->os->get_current_image_name()
>
> based on the desired return value, then node_status or other routines
> can then compare what is loaded vs what is requested.
>
> and lastest remove the old utils::_getcurrentimage routine
>
> This would allow for changes in the name like shorter and less strict
> naming of the internal imagenames.
>
> Thoughts or comments before I proceed?
>
> Aaron P.
>
>
> On Fri, Sep 21, 2012 at 11:07 AM, Aaron Peeler (JIRA) <ji...@apache.org> wrote:
>>
>> [ https://issues.apache.org/jira/browse/VCL-630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460545#comment-13460545 ]
>>
>> Aaron Peeler commented on VCL-630:
>> ----------------------------------
>>
>> follow-up - there is a get_currentimage_txt_contents in the OS.pm module already. I think the best path would be to create a OS::get_currentimage_imagerevision_id routine which parses the contents and returns the imagerevisionid on the loaded node. Then update the provisioning module's node_status routines to use this instead.
>>
>>> currentimage.txt name may conflict with imagerevision name, causing unnecessary reloads
>>> ---------------------------------------------------------------------------------------
>>>
>>> Key: VCL-630
>>> URL: https://issues.apache.org/jira/browse/VCL-630
>>> Project: VCL
>>> Issue Type: Bug
>>> Components: vcld (backend)
>>> Affects Versions: 2.3
>>> Environment: accessing the first revision of an image on a clustered (vCenter) VMware host. This only affects images whose name (cleaned of non-alphanumeric characters) exceeds 29 characters.
>>> Reporter: Aaron Coburn
>>> Assignee: Aaron Coburn
>>> Fix For: 2.4, 2.3.1
>>>
>>>
>>> A few words to provide some background: when the VCL uses a clustered (vCenter) VMware host, one cannot simply copy virtual disks as part of the image capture process; instead, the VM is cloned. This, however, exposed an apparently undocumented feature of VMware in that virtual disk names exceeding 29 characters are silently truncated. This creates significant problems for the VCL. The current solution (for 2.3) is to update the image name in the database after successfully capturing an image. This way, the VCL knows where to find the virtual disk in the VMware datastore.
>>> The problem with this approach is that this change isn't registered with the system until after the image has been captured. At this point, however, the currentimage.txt file has already been written to the disk and if the name changes ex post facto, then the problem simply re-emerges in a different form when a user makes a reservation for the image.
>>> Assuming the VM is available and otherwise ready to grant an immediate reservation, the reclaim.pm:process() subroutine checks the VM name listed in the currentimage.txt file and finds a conflict in the name listed on the OS and the name in the database.
>>> An immediate solution for users is to simply make a new revision of the same image. That second revision will have a shortened name in both currentimage.txt and the imagerevision table.
>>> There are two ways that this could be correctly solved in the perl code: either by updating the image name earlier in the provisioning module's capture() subroutine or by extracting other information from currentimage.txt, such as imagerevision_id.
>>> While the first approach would be easier to implement, the second approach seems to be more architecturally sound. That is, rather than relying on a comparison of two strings, the test would be between two integers, one of which is the primary key of the database table in question.
>>> Any thoughts on the best way to proceed?
>>
>> --
>> This message is automatically generated by JIRA.
>> If you think it was sent incorrectly, please contact your JIRA administrators
>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
> --
> Aaron Peeler
> Program Manager
> Virtual Computing Lab
> NC State University
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
--
Mark Gardner
--