You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@vcl.apache.org by Aaron Peeler <aa...@ncsu.edu> on 2009/08/04 17:51:57 UTC
Re: VCL questions (provisioning/deprovisioning)
Toks,
Yes this is true for any compute resource - bare-metal or virtual machine.
The goal being to have that compute resource reloaded and ready for the
next user. In an edu environment - students, faculty have a hard time
waiting more than a minute for anything.
There is a jira issue on power-management. The general idea behind this is
defining a policy on when to power-down compute resources, be it a blade or
vm. Which the policy could be unload all the time or when the overall
system is less than a certain percentage of use.
But in the short-term this unload option could be part of the provisioning
module load routine.
By making use of the data-structure hash, one could find out the last-state
of the compute resource i.e if 'inuse' then call an unload routine,
powerdown and then make the state of the machine available or release it.
Hope this is what your looking for.
Aaron
> Andy,
>
> I would like to revisit this discussion. I believe I understand how this
> is applicable when physical resources are being provisioned. Does the
> same hold true for virtual resources? It seems that once a VM is
> provisioned through the vmware or esx modules, that it would persist
> indefinitely either via reloading or sanitization for as long as it's
> host system is up and running, waiting for another user to "claim" it. I
> haven't observed a case where a provisioned virtual resource's lifecycle
> actually comes to an end.
>
> Toks
>>>>
>>>> Toks
>>>>
>
Aaron Peeler
OIT Advanced Computing
College of Engineering-NCSU
919.513.4571
http://vcl.ncsu.edu
RE: VCL image reservation
Posted by Kelly Robinson <is...@langate.gsu.edu>.
Would this contribute to the error message in the vcld.log of "image already exists in repository"?
log output snippet:
====================================
.
.
.
2009-09-28 11:31:35|375|30:99|image|Module.pm:new(80)|constructor called, class=VCL::Module::OS::Windows
2009-09-28 11:31:35|375|30:99|image|Module.pm:new(102)|VCL::Module::OS::Windows object created
2009-09-28 11:31:35|375|30:99|image|State.pm:initialize(155)|VCL::Module::OS::Windows OS object successfully created
2009-09-28 11:31:35|375|30:99|image|State.pm:initialize(170)|returning 1
2009-09-28 11:31:35|375|30:99|image|vcld:make_new_child(597)|VCL::image object created and initialized
/usr/lib/sendmail: option requires an argument -- f
2009-09-28 11:31:35|375|30:99|image|utils.pm:mail(1301)|SUCCESS -- Sending mail To: , VCL IMAGE Creation Started: vmwareinxp-base7-v1
2009-09-28 11:31:35|375|30:99|image|utils.pm:run_ssh_command(6695)|executing SSH command on 131.96.10.178:
|375|30:99|image| ls -1 /install/vmware_images/golden 2>&1
2009-09-28 11:31:35|375|30:99|image|utils.pm:run_ssh_command(6776)|run_ssh_command output:
|375|30:99|image| vmwareinxp-base7-v1
2009-09-28 11:31:35|375|30:99|image|utils.pm:run_ssh_command(6784)|SSH command executed on 131.96.10.178, returning (0, "vmwareinxp-base7-v1 vmwareinxp...")
2009-09-28 11:31:35|375|30:99|image|esx.pm:does_image_exist(775)|image vmwareinxp-base7-v1 exists
/usr/lib/sendmail: option requires an argument -- f
2009-09-28 11:31:35|375|30:99|image|utils.pm:mail(1301)|SUCCESS -- Sending mail To: , PROBLEM -- image.pm
|375|30:99|image| ---- CRITICAL ----
|375|30:99|image| 2009-09-28 11:31:35|375|30:99|image|image.pm:process(137)|image vmwareinxp-base7-v1 already exists in the repository
|375|30:99|image| ( 0) utils.pm, notify (line: 691)
|375|30:99|image| (-1) image.pm, process (line: 137)
|375|30:99|image| (-2) vcld, make_new_child (line: 600)
|375|30:99|image| (-3) vcld, main (line: 347)
/usr/lib/sendmail: option requires an argument -- f
2009-09-28 11:31:35|375|30:99|image|utils.pm:mail(1301)|SUCCESS -- Sending mail To: , PROBLEM -- image.pm
|375|30:99|image| ---- CRITICAL ----
|375|30:99|image| 2009-09-28 11:31:35|375|30:99|image|image.pm:reservation_failed(390)|vmwareinxp-base7-v1 image creation failed
|375|30:99|image| ( 0) utils.pm, notify (line: 691)
|375|30:99|image| (-1) image.pm, reservation_failed (line: 390)
|375|30:99|image| (-2) image.pm, process (line: 138)
|375|30:99|image| (-3) vcld, make_new_child (line: 600)
|375|30:99|image| (-4) vcld, main (line: 347)
/usr/lib/sendmail: option requires an argument -- f
2009-09-28 11:31:35|375|30:99|image|utils.pm:mail(1301)|SUCCESS -- Sending mail To: root@localhost, VCL -- NOTICE DELAY Image Creation No Apps (WinXP vmware)
/usr/lib/sendmail: option requires an argument -- f
2009-09-28 11:31:35|375|30:99|image|utils.pm:mail(1301)|SUCCESS -- Sending mail To: , VCL -- NOTICE FAILED Image Creation No Apps (WinXP vmware)
2009-09-28 11:31:35|375|30:99|image|utils.pm:update_request_state(2283)|request 30 state updated to: maintenance, laststate to: image
2009-09-28 11:31:35|375|30:99|image|image.pm:reservation_failed(442)|request state set to maintenance, laststate to image
2009-09-28 11:31:35|375|30:99|image|utils.pm:update_computer_state(2325)|computer 2 state updated to: maintenance
2009-09-28 11:31:35|375|30:99|image|image.pm:reservation_failed(450)|vclnode005 state set to maintenance
2009-09-28 11:31:35|375|30:99|image|image.pm:reservation_failed(457)|exiting
2009-09-28 11:31:35|375|30:99|image|State.pm:DESTROY(442)|destructor called, ref($self)=VCL::image
2009-09-28 11:31:35|375|30:99|image|utils.pm:delete_computerloadlog_reservation(8041)|removing computerloadlog entries matching loadstate = begin
2009-09-28 11:31:35|375|30:99|image|utils.pm:delete_computerloadlog_reservation(8088)|deleted rows from computerloadlog for reservation id=99
2009-09-28 11:31:35|375|30:99|image|State.pm:DESTROY(445)|removed computerloadlog rows with loadstate=begin for reservation
2009-09-28 11:31:35|375|30:99|image|State.pm:DESTROY(453)|number of database handles state process created: 1
2009-09-28 11:31:35|375|30:99|image|State.pm:DESTROY(462)|process has a database handle stored in $ENV{dbh}, attempting disconnect
2009-09-28 11:31:35|375|30:99|image|State.pm:DESTROY(464)|$ENV{dbh}: database disconnect successful
===============================================
Kelly
>>> "Miller, Fred Anthony" <MI...@ecu.edu> 09/28/2009 11:01 AM >>>
In addition to adding the image to the database, you have to ensure that you have the image in a group, that the group is mapped to valid computer(s), and that there is a privilege node that includes the image group, the computer group, and your user with the privilege imagecheckout.
Step by step, you want to
1) Create a computer group with one or more of your blades/vms (depending on whether the image is bare metal or a VM). In the web interface, you want to use Manage Groups and create a new computer group in the "Resource Groups" section.
2) Create an image group to contain your image. In the web interface, you want to use Manage Groups and create a new image group in the "Resource Groups" section
3) Add blades or vm computers (depending on type of image) to the computer group. In the web interface, that's "Manage Computers" -> "Edit Computer Grouping" radio button -> "By Group" tab. That interface lets you add existing computers to your new computer group.
4) Add your inserted image into an image group. In the web interface, that's "Manage Images" -> "Edit Image Grouping" radio button -> "By Group" tab. That interface lets you add existing images to your new image group.
5) Map your image group to the computer gropu. In the web interface, that's "Manage Image" -> "Edit Image Mapping" radio button -> "by Image Group" tab. You can then pick from the dropdown your new image group, click "Get Computer Groups" button, and then add your new computer group.
6) Create or update privilege node. In the web interface, that's "Privileges". If you have a node, you can add the image group and computer group as available resources. If you don't, create a new node, then add the computer group and image group to the "Resources" section. Make sure that they at least have the "available" privilege. Add your individual user or a user group you are a member of to the "Users" section. Make sure that the imageCheckOut privilege is checked.
You can do these things at the database level, too, but it requires a fair amount of lookups and inserts. I've done computer and image groups manually, but not privilege nodes.
PS: If you've done all this, make sure that the computers mapped to the image are not all in "reload", or "inuse" state. If all the machines are not "available" or "failed", you won't be able to make a reservation.
--
Tony Miller
Technology Consultant
Academic Technologies
millerf@ecu.edu
-----Original Message-----
From: Kelly Robinson [mailto:ISGKPR@langate.gsu.edu]
Sent: Sunday, September 27, 2009 10:15 PM
To: vcl-dev@incubator.apache.org
Subject: VCL image reservation
How do you get VCL to allow an image to be requested? I've added the image to the database and would now like to request it through the web interface, however when you click "New Reservation" and select the image from the drop down menu, it gives a message that states "Selection Not Currently Available".
I've tried to manually insert a request into the database using the following sql statements, however the vcld.log gives error messages stating that the image already exists in the repository. What am I missing?
=================================
INSERT INTO `vcl`.`request` (
`id` ,
`stateid` ,
`userid` ,
`laststateid` ,
`logid` ,
`forimaging` ,
`test` ,
`preload` ,
`start` ,
`end` ,
`daterequested` ,
`datemodified`
)
VALUES (
'1' , '16', '1', '16', '', '1', '0', '0', NOW( ) , TIMESTAMPADD(MINUTE, 120, NOW()), NOW( ) , NULL
);
===============================================
===============================================
INSERT INTO `vcl`.`reservation` (
`id` ,
`requestid` ,
`computerid` ,
`imageid` ,
`imagerevisionid` ,
`managementnodeid` ,
`remoteIP` ,
`lastcheck` ,
`pw`
)
VALUES (
NULL , '1', '2', '7', '7', '1', NULL , NULL , NULL
);
===========================================
Thanks,
Kelly
RE: VCL image reservation
Posted by "Miller, Fred Anthony" <MI...@ecu.edu>.
In addition to adding the image to the database, you have to ensure that you have the image in a group, that the group is mapped to valid computer(s), and that there is a privilege node that includes the image group, the computer group, and your user with the privilege imagecheckout.
Step by step, you want to
1) Create a computer group with one or more of your blades/vms (depending on whether the image is bare metal or a VM). In the web interface, you want to use Manage Groups and create a new computer group in the "Resource Groups" section.
2) Create an image group to contain your image. In the web interface, you want to use Manage Groups and create a new image group in the "Resource Groups" section
3) Add blades or vm computers (depending on type of image) to the computer group. In the web interface, that's "Manage Computers" -> "Edit Computer Grouping" radio button -> "By Group" tab. That interface lets you add existing computers to your new computer group.
4) Add your inserted image into an image group. In the web interface, that's "Manage Images" -> "Edit Image Grouping" radio button -> "By Group" tab. That interface lets you add existing images to your new image group.
5) Map your image group to the computer gropu. In the web interface, that's "Manage Image" -> "Edit Image Mapping" radio button -> "by Image Group" tab. You can then pick from the dropdown your new image group, click "Get Computer Groups" button, and then add your new computer group.
6) Create or update privilege node. In the web interface, that's "Privileges". If you have a node, you can add the image group and computer group as available resources. If you don't, create a new node, then add the computer group and image group to the "Resources" section. Make sure that they at least have the "available" privilege. Add your individual user or a user group you are a member of to the "Users" section. Make sure that the imageCheckOut privilege is checked.
You can do these things at the database level, too, but it requires a fair amount of lookups and inserts. I've done computer and image groups manually, but not privilege nodes.
PS: If you've done all this, make sure that the computers mapped to the image are not all in "reload", or "inuse" state. If all the machines are not "available" or "failed", you won't be able to make a reservation.
--
Tony Miller
Technology Consultant
Academic Technologies
millerf@ecu.edu
-----Original Message-----
From: Kelly Robinson [mailto:ISGKPR@langate.gsu.edu]
Sent: Sunday, September 27, 2009 10:15 PM
To: vcl-dev@incubator.apache.org
Subject: VCL image reservation
How do you get VCL to allow an image to be requested? I've added the image to the database and would now like to request it through the web interface, however when you click "New Reservation" and select the image from the drop down menu, it gives a message that states "Selection Not Currently Available".
I've tried to manually insert a request into the database using the following sql statements, however the vcld.log gives error messages stating that the image already exists in the repository. What am I missing?
=================================
INSERT INTO `vcl`.`request` (
`id` ,
`stateid` ,
`userid` ,
`laststateid` ,
`logid` ,
`forimaging` ,
`test` ,
`preload` ,
`start` ,
`end` ,
`daterequested` ,
`datemodified`
)
VALUES (
'1' , '16', '1', '16', '', '1', '0', '0', NOW( ) , TIMESTAMPADD(MINUTE, 120, NOW()), NOW( ) , NULL
);
===============================================
===============================================
INSERT INTO `vcl`.`reservation` (
`id` ,
`requestid` ,
`computerid` ,
`imageid` ,
`imagerevisionid` ,
`managementnodeid` ,
`remoteIP` ,
`lastcheck` ,
`pw`
)
VALUES (
NULL , '1', '2', '7', '7', '1', NULL , NULL , NULL
);
===========================================
Thanks,
Kelly
VCL image reservation
Posted by Kelly Robinson <IS...@langate.gsu.edu>.
How do you get VCL to allow an image to be requested? I've added the image to the database and would now like to request it through the web interface, however when you click "New Reservation" and select the image from the drop down menu, it gives a message that states "Selection Not Currently Available".
I've tried to manually insert a request into the database using the following sql statements, however the vcld.log gives error messages stating that the image already exists in the repository. What am I missing?
=================================
INSERT INTO `vcl`.`request` (
`id` ,
`stateid` ,
`userid` ,
`laststateid` ,
`logid` ,
`forimaging` ,
`test` ,
`preload` ,
`start` ,
`end` ,
`daterequested` ,
`datemodified`
)
VALUES (
'1' , '16', '1', '16', '', '1', '0', '0', NOW( ) , TIMESTAMPADD(MINUTE, 120, NOW()), NOW( ) , NULL
);
===============================================
===============================================
INSERT INTO `vcl`.`reservation` (
`id` ,
`requestid` ,
`computerid` ,
`imageid` ,
`imagerevisionid` ,
`managementnodeid` ,
`remoteIP` ,
`lastcheck` ,
`pw`
)
VALUES (
NULL , '1', '2', '7', '7', '1', NULL , NULL , NULL
);
===========================================
Thanks,
Kelly
Re: VCL questions (provisioning/deprovisioning)
Posted by Toks Adeshiyan <to...@linux.vnet.ibm.com>.
Thanks Aaron. This explanation helps a whole lot -- I understand this
issue much better.
Toks
Aaron Peeler wrote:
> Toks,
>
> Yes this is true for any compute resource - bare-metal or virtual
> machine.
>
> The goal being to have that compute resource reloaded and ready for
> the next user. In an edu environment - students, faculty have a hard
> time waiting more than a minute for anything.
>
> There is a jira issue on power-management. The general idea behind
> this is defining a policy on when to power-down compute resources, be
> it a blade or vm. Which the policy could be unload all the time or
> when the overall system is less than a certain percentage of use.
>
> But in the short-term this unload option could be part of the
> provisioning module load routine.
> By making use of the data-structure hash, one could find out the
> last-state of the compute resource i.e if 'inuse' then call an unload
> routine, powerdown and then make the state of the machine available or
> release it.
>
> Hope this is what your looking for.
>
> Aaron
>
>> Andy,
>>
>> I would like to revisit this discussion. I believe I understand how
>> this
>> is applicable when physical resources are being provisioned. Does the
>> same hold true for virtual resources? It seems that once a VM is
>> provisioned through the vmware or esx modules, that it would persist
>> indefinitely either via reloading or sanitization for as long as it's
>> host system is up and running, waiting for another user to "claim"
>> it. I
>> haven't observed a case where a provisioned virtual resource's lifecycle
>> actually comes to an end.
>>
>> Toks
>>>>>
>>>>> Toks
>>>>>
>>
>
>
>
> Aaron Peeler
> OIT Advanced Computing
> College of Engineering-NCSU
> 919.513.4571
> http://vcl.ncsu.edu