You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@vcl.apache.org by "Madarasz, Norbert" <n....@gmail.com> on 2013/04/15 08:29:34 UTC
Logic of allocating computer to reservation + VCL on Zend Server
Hi all,
This is the first time to write you. As part of my master's thesis at
Budapest University I should simulate the key functions of VCL with
CloudSim.
For that the allocating logic is very important part means how a computer
is assigned to a reservation. I checked the cwiki page of VCL but the exact
procedure is not published. I've started to look inside the code and found
the predictive levels in PERL modules and it was understandable thanks to
vcld.log file which helped me to backtrace when, how and who called thoose
functions.
Unfortunately this is not enough because the logic is missing how a
computer will be assigned to an image/reservation. As I saw this allocating
happens in Request.php (fn newReservation()) and maybe the main logic is in
Utils.php (fn allocComputer). My problem is that there is no log that I can
use to backtrace what and when happening and I don't understand how this
allocating works. Some questions: Is it random which available computer
will be used? If a computer is preloaded with the same image that a user
chose too then where and how will be taken this into account and filter
that computers which are preloaded with that image? What happens if a user
requests such an image that is not preloaded on any computer, is it problem
or only the provisioning time period will last a bit more?
Can someone explain the logic behind the allocating procedure and tell me
how the "computer-id" value of reservations table is calculated?
Can you recommend me some technique to trace the "happenings" in php
scripts? I tried to use Zend Server instead of php5 server but there is
something wrong. Have you tried to use it with Zend Server, any
documentation or it shouldn't be problem?
Thank you in advance!
Best regards,
Norbert
My advisor is Imre Kocsis and he suggested to mail here:
"Hi all,
I am Norbert's advisor. Norbert, I think you should have posted this to the
'dev' list rather than 'user'. Anyways, back in the summer when we started
getting our feet wet with VCL I managed to find only a short and incomplete
thread on the mailing list explaining the (largely undocumented) machine
allocation and preparation scheduling logic - so any help is greatly
appreciated. (Although based on our experience with the code base so far I
think the necessary information can be traced back from the source with
manageable effort.)
To offer a bit of context to Norbert's questions: the larger goal is to
have a simple discrete event simulator for a hybrid VCL setup in order to
be able to play around with various reservation arrival processes, cost and
possibly fault models. (I know that 'hybrid' VCL setups - as in mixing an
in-house data center e.g. with EC2 - is not a technical reality for VCL
yet, but we are very interested in how it could work from the long-term
cost optimization and risk mitigation point of view for 'hardware budget
conscious' VCL setups.)
After understanding the necessary parts of VCL, hopefully we will be able
to contribute some sort of documentation on (our understanding of) the
internals back to the community.
Best regards
Imre"
Re: Logic of allocating computer to reservation + VCL on Zend Server
Posted by "Madarasz, Norbert" <n....@gmail.com>.
Thanks Aaron and Josh, Josh's answer was the answer I was looking for.
On 16 April 2013 14:41, Josh Thompson <jo...@ncsu.edu> wrote:
> On Tuesday, April 16, 2013 12:04:37 PM Aaron Coburn wrote:
> > Hi, Norbert,
> >
> > > Further question1: In fn isAvailable there is a part to find computers
> > > whose hosts can handle the required RAM. Would not be necessary to have
> > > something same for CPU?
> >
> > The same is done for CPU count, CPU speed and network speed. You will
> notice
> > in the query that the results will only include computers with specs
> > greater than or equal to the image requirements.
>
> There are two ways RAM is involved in selection. One is ranking the
> computers
> for selection; the other is a RAM check only used for virtual machines.
> There
> is an additional section in isAvailable where, if there are no preloaded
> VMs,
> there is a check to only select a VM if the host can allocate the RAM that
> would be required for the VM. If the latter portion is the part Norbert is
> referring to, he is correct that there is not a corresponding check for CPU
> utilization. The reason for this is we've tended to overbook CPU by
> varying
> amounts.
>
> Josh
>
> > > FQ2: You mentioned ranking will be reversed (say Order by Asc in SQL
> query
> > > part of
> > >
> #image.OSid->OS.installtype->OSinstalltype.id->provisioningOSinstalltype.p
> > > rovisioningid->computer.provisioningid) in the next major release so
> that
> > > computers
> > > with the lowest specs will be ranked first. Is it better to keep the
> > > computers with harder specs for future requests?
> >
> > There was a lengthier discussion of this on the dev list a few months ago
> > [1] with some additional background.
> >
> > Best regards,
> > Aaron
> >
> >
> > [1] http://markmail.org/thread/d7jgat3buoah4mhe
> >
> >
> >
> > --
> > Aaron Coburn
> > Systems Administrator and Programmer
> > Academic Technology Services, Amherst College
> > acoburn@amherst.edu
> >
> > > Thanks in advance,
> > > Norbert
> > >
> > > On 15 April 2013 15:01, Josh Thompson <jo...@ncsu.edu> wrote:
> > >> Norbert,
> > >>
> > >> Answers are inline.
> > >>
> > >> On Monday, April 15, 2013 8:29:34 AM Madarasz, Norbert wrote:
> > >>> Hi all,
> > >>>
> > >>> This is the first time to write you. As part of my master's thesis at
> > >>> Budapest University I should simulate the key functions of VCL with
> > >>> CloudSim.
> > >>>
> > >>> For that the allocating logic is very important part means how a
> > >>> computer
> > >>> is assigned to a reservation. I checked the cwiki page of VCL but the
> > >>
> > >> exact
> > >>
> > >>> procedure is not published. I've started to look inside the code and
> > >>
> > >> found
> > >>
> > >>> the predictive levels in PERL modules and it was understandable
> thanks
> > >>> to
> > >>> vcld.log file which helped me to backtrace when, how and who called
> > >>
> > >> thoose
> > >>
> > >>> functions.
> > >>>
> > >>> Unfortunately this is not enough because the logic is missing how a
> > >>> computer will be assigned to an image/reservation. As I saw this
> > >>
> > >> allocating
> > >>
> > >>> happens in Request.php (fn newReservation()) and maybe the main
> logic is
> > >>
> > >> in
> > >>
> > >>> Utils.php (fn allocComputer).
> > >>
> > >> You need to look at the isAvailable function in utils.php. That's
> where
> > >> a
> > >> computer is selected for any given reservation.
> > >>
> > >>> My problem is that there is no log that I can
> > >>> use to backtrace what and when happening and I don't understand how
> this
> > >>> allocating works.
> > >>
> > >> Correct - there is no log of how the computers are allocated. You can
> > >> use
> > >> the
> > >> php function "error_log" to add your own logging to the isAvailable
> > >> function
> > >> to see what is happening.
> > >>
> > >>> Some questions: Is it random which available computer
> > >>> will be used? If a computer is preloaded with the same image that a
> user
> > >>> chose too then where and how will be taken this into account and
> filter
> > >>> that computers which are preloaded with that image?
> > >>
> > >> By default, it is not random. Available computers are ordered first
> by
> > >> whether or not they are currently loaded with the requested image,
> then
> > >> by
> > >> the
> > >> specs of the computer with computers having the highest specs ranked
> > >> first
> > >> (however, that will be reversed in the next major release so that
> > >> computers
> > >> with the lowest specs will be ranked first).
> > >>
> > >> There is a config option in the conf.php file that will change the
> > >> behavior so
> > >> that computers are randomly assigned rather than being ranked by specs
> > >> (they
> > >> are still separated by preloaded/not preloaded).
> > >>
> > >>> What happens if a user
> > >>> requests such an image that is not preloaded on any computer, is it
> > >>
> > >> problem
> > >>
> > >>> or only the provisioning time period will last a bit more?
> > >>
> > >> If a computer is not preloaded, it just makes the provisioning time
> > >> longer.
> > >>
> > >>> Can someone explain the logic behind the allocating procedure and
> tell
> > >>> me
> > >>> how the "computer-id" value of reservations table is calculated?
> > >>
> > >> See above.
> > >>
> > >>> Can you recommend me some technique to trace the "happenings" in php
> > >>> scripts? I tried to use Zend Server instead of php5 server but there
> is
> > >>> something wrong. Have you tried to use it with Zend Server, any
> > >>> documentation or it shouldn't be problem?
> > >>
> > >> I've never used Zend Server. I'd recommend using the error_log
> function
> > >> as
> > >> described above.
> > >>
> > >>> Thank you in advance!
> > >>
> > >> You're welcome.
> > >>
> > >>> Best regards,
> > >>> Norbert
> > >>
> > >> Josh
> > >>
> > >>> My advisor is Imre Kocsis and he suggested to mail here:
> > >>> "Hi all,
> > >>>
> > >>> I am Norbert's advisor. Norbert, I think you should have posted this
> to
> > >>
> > >> the
> > >>
> > >>> 'dev' list rather than 'user'. Anyways, back in the summer when we
> > >>
> > >> started
> > >>
> > >>> getting our feet wet with VCL I managed to find only a short and
> > >>
> > >> incomplete
> > >>
> > >>> thread on the mailing list explaining the (largely undocumented)
> machine
> > >>> allocation and preparation scheduling logic - so any help is greatly
> > >>> appreciated. (Although based on our experience with the code base so
> far
> > >>
> > >> I
> > >>
> > >>> think the necessary information can be traced back from the source
> with
> > >>> manageable effort.)
> > >>>
> > >>> To offer a bit of context to Norbert's questions: the larger goal is
> to
> > >>> have a simple discrete event simulator for a hybrid VCL setup in
> order
> > >>> to
> > >>> be able to play around with various reservation arrival processes,
> cost
> > >>
> > >> and
> > >>
> > >>> possibly fault models. (I know that 'hybrid' VCL setups - as in
> mixing
> > >>> an
> > >>> in-house data center e.g. with EC2 - is not a technical reality for
> VCL
> > >>> yet, but we are very interested in how it could work from the
> long-term
> > >>> cost optimization and risk mitigation point of view for 'hardware
> budget
> > >>> conscious' VCL setups.)
> > >>>
> > >>> After understanding the necessary parts of VCL, hopefully we will be
> > >>> able
> > >>> to contribute some sort of documentation on (our understanding of)
> the
> > >>> internals back to the community.
> > >>>
> > >>> Best regards
> > >>> Imre"
> > >>
> > >> --
> > >> -------------------------------
> > >> Josh Thompson
> > >> Systems Programmer
> > >> Advanced Computing | VCL Developer
> > >> North Carolina State University
> > >>
> > >> Josh_Thompson@ncsu.edu
> > >> 919-515-5323
> > >>
> > >> my GPG/PGP key can be found at pgp.mit.edu
> > >>
> > >> All electronic mail messages in connection with State business which
> > >> are sent to or received by this account are subject to the NC Public
> > >> Records Law and may be disclosed to third parties.
> --
> -------------------------------
> Josh Thompson
> Systems Programmer
> Advanced Computing | VCL Developer
> North Carolina State University
>
> Josh_Thompson@ncsu.edu
> 919-515-5323
>
> my GPG/PGP key can be found at pgp.mit.edu
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
>
Re: Logic of allocating computer to reservation + VCL on Zend Server
Posted by Josh Thompson <jo...@ncsu.edu>.
On Tuesday, April 16, 2013 12:04:37 PM Aaron Coburn wrote:
> Hi, Norbert,
>
> > Further question1: In fn isAvailable there is a part to find computers
> > whose hosts can handle the required RAM. Would not be necessary to have
> > something same for CPU?
>
> The same is done for CPU count, CPU speed and network speed. You will notice
> in the query that the results will only include computers with specs
> greater than or equal to the image requirements.
There are two ways RAM is involved in selection. One is ranking the computers
for selection; the other is a RAM check only used for virtual machines. There
is an additional section in isAvailable where, if there are no preloaded VMs,
there is a check to only select a VM if the host can allocate the RAM that
would be required for the VM. If the latter portion is the part Norbert is
referring to, he is correct that there is not a corresponding check for CPU
utilization. The reason for this is we've tended to overbook CPU by varying
amounts.
Josh
> > FQ2: You mentioned ranking will be reversed (say Order by Asc in SQL query
> > part of
> > #image.OSid->OS.installtype->OSinstalltype.id->provisioningOSinstalltype.p
> > rovisioningid->computer.provisioningid) in the next major release so that
> > computers
> > with the lowest specs will be ranked first. Is it better to keep the
> > computers with harder specs for future requests?
>
> There was a lengthier discussion of this on the dev list a few months ago
> [1] with some additional background.
>
> Best regards,
> Aaron
>
>
> [1] http://markmail.org/thread/d7jgat3buoah4mhe
>
>
>
> --
> Aaron Coburn
> Systems Administrator and Programmer
> Academic Technology Services, Amherst College
> acoburn@amherst.edu
>
> > Thanks in advance,
> > Norbert
> >
> > On 15 April 2013 15:01, Josh Thompson <jo...@ncsu.edu> wrote:
> >> Norbert,
> >>
> >> Answers are inline.
> >>
> >> On Monday, April 15, 2013 8:29:34 AM Madarasz, Norbert wrote:
> >>> Hi all,
> >>>
> >>> This is the first time to write you. As part of my master's thesis at
> >>> Budapest University I should simulate the key functions of VCL with
> >>> CloudSim.
> >>>
> >>> For that the allocating logic is very important part means how a
> >>> computer
> >>> is assigned to a reservation. I checked the cwiki page of VCL but the
> >>
> >> exact
> >>
> >>> procedure is not published. I've started to look inside the code and
> >>
> >> found
> >>
> >>> the predictive levels in PERL modules and it was understandable thanks
> >>> to
> >>> vcld.log file which helped me to backtrace when, how and who called
> >>
> >> thoose
> >>
> >>> functions.
> >>>
> >>> Unfortunately this is not enough because the logic is missing how a
> >>> computer will be assigned to an image/reservation. As I saw this
> >>
> >> allocating
> >>
> >>> happens in Request.php (fn newReservation()) and maybe the main logic is
> >>
> >> in
> >>
> >>> Utils.php (fn allocComputer).
> >>
> >> You need to look at the isAvailable function in utils.php. That's where
> >> a
> >> computer is selected for any given reservation.
> >>
> >>> My problem is that there is no log that I can
> >>> use to backtrace what and when happening and I don't understand how this
> >>> allocating works.
> >>
> >> Correct - there is no log of how the computers are allocated. You can
> >> use
> >> the
> >> php function "error_log" to add your own logging to the isAvailable
> >> function
> >> to see what is happening.
> >>
> >>> Some questions: Is it random which available computer
> >>> will be used? If a computer is preloaded with the same image that a user
> >>> chose too then where and how will be taken this into account and filter
> >>> that computers which are preloaded with that image?
> >>
> >> By default, it is not random. Available computers are ordered first by
> >> whether or not they are currently loaded with the requested image, then
> >> by
> >> the
> >> specs of the computer with computers having the highest specs ranked
> >> first
> >> (however, that will be reversed in the next major release so that
> >> computers
> >> with the lowest specs will be ranked first).
> >>
> >> There is a config option in the conf.php file that will change the
> >> behavior so
> >> that computers are randomly assigned rather than being ranked by specs
> >> (they
> >> are still separated by preloaded/not preloaded).
> >>
> >>> What happens if a user
> >>> requests such an image that is not preloaded on any computer, is it
> >>
> >> problem
> >>
> >>> or only the provisioning time period will last a bit more?
> >>
> >> If a computer is not preloaded, it just makes the provisioning time
> >> longer.
> >>
> >>> Can someone explain the logic behind the allocating procedure and tell
> >>> me
> >>> how the "computer-id" value of reservations table is calculated?
> >>
> >> See above.
> >>
> >>> Can you recommend me some technique to trace the "happenings" in php
> >>> scripts? I tried to use Zend Server instead of php5 server but there is
> >>> something wrong. Have you tried to use it with Zend Server, any
> >>> documentation or it shouldn't be problem?
> >>
> >> I've never used Zend Server. I'd recommend using the error_log function
> >> as
> >> described above.
> >>
> >>> Thank you in advance!
> >>
> >> You're welcome.
> >>
> >>> Best regards,
> >>> Norbert
> >>
> >> Josh
> >>
> >>> My advisor is Imre Kocsis and he suggested to mail here:
> >>> "Hi all,
> >>>
> >>> I am Norbert's advisor. Norbert, I think you should have posted this to
> >>
> >> the
> >>
> >>> 'dev' list rather than 'user'. Anyways, back in the summer when we
> >>
> >> started
> >>
> >>> getting our feet wet with VCL I managed to find only a short and
> >>
> >> incomplete
> >>
> >>> thread on the mailing list explaining the (largely undocumented) machine
> >>> allocation and preparation scheduling logic - so any help is greatly
> >>> appreciated. (Although based on our experience with the code base so far
> >>
> >> I
> >>
> >>> think the necessary information can be traced back from the source with
> >>> manageable effort.)
> >>>
> >>> To offer a bit of context to Norbert's questions: the larger goal is to
> >>> have a simple discrete event simulator for a hybrid VCL setup in order
> >>> to
> >>> be able to play around with various reservation arrival processes, cost
> >>
> >> and
> >>
> >>> possibly fault models. (I know that 'hybrid' VCL setups - as in mixing
> >>> an
> >>> in-house data center e.g. with EC2 - is not a technical reality for VCL
> >>> yet, but we are very interested in how it could work from the long-term
> >>> cost optimization and risk mitigation point of view for 'hardware budget
> >>> conscious' VCL setups.)
> >>>
> >>> After understanding the necessary parts of VCL, hopefully we will be
> >>> able
> >>> to contribute some sort of documentation on (our understanding of) the
> >>> internals back to the community.
> >>>
> >>> Best regards
> >>> Imre"
> >>
> >> --
> >> -------------------------------
> >> Josh Thompson
> >> Systems Programmer
> >> Advanced Computing | VCL Developer
> >> North Carolina State University
> >>
> >> Josh_Thompson@ncsu.edu
> >> 919-515-5323
> >>
> >> my GPG/PGP key can be found at pgp.mit.edu
> >>
> >> All electronic mail messages in connection with State business which
> >> are sent to or received by this account are subject to the NC Public
> >> Records Law and may be disclosed to third parties.
--
-------------------------------
Josh Thompson
Systems Programmer
Advanced Computing | VCL Developer
North Carolina State University
Josh_Thompson@ncsu.edu
919-515-5323
my GPG/PGP key can be found at pgp.mit.edu
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.
Re: Logic of allocating computer to reservation + VCL on Zend Server
Posted by Aaron Coburn <ac...@amherst.edu>.
Hi, Norbert,
> Further question1: In fn isAvailable there is a part to find computers
> whose hosts can handle the required RAM. Would not be necessary to have
> something same for CPU?
The same is done for CPU count, CPU speed and network speed. You will notice in the query that the results will only include computers with specs greater than or equal to the image requirements.
> FQ2: You mentioned ranking will be reversed (say Order by Asc in SQL query
> part of
> #image.OSid->OS.installtype->OSinstalltype.id->provisioningOSinstalltype.provisioningid->computer.provisioningid)
> in the next major release so that computers
> with the lowest specs will be ranked first. Is it better to keep the
> computers with harder specs for future requests?
There was a lengthier discussion of this on the dev list a few months ago [1] with some additional background.
Best regards,
Aaron
[1] http://markmail.org/thread/d7jgat3buoah4mhe
--
Aaron Coburn
Systems Administrator and Programmer
Academic Technology Services, Amherst College
acoburn@amherst.edu
> Thanks in advance,
> Norbert
>
>
> On 15 April 2013 15:01, Josh Thompson <jo...@ncsu.edu> wrote:
>
>> Norbert,
>>
>> Answers are inline.
>>
>> On Monday, April 15, 2013 8:29:34 AM Madarasz, Norbert wrote:
>>> Hi all,
>>>
>>> This is the first time to write you. As part of my master's thesis at
>>> Budapest University I should simulate the key functions of VCL with
>>> CloudSim.
>>>
>>> For that the allocating logic is very important part means how a computer
>>> is assigned to a reservation. I checked the cwiki page of VCL but the
>> exact
>>> procedure is not published. I've started to look inside the code and
>> found
>>> the predictive levels in PERL modules and it was understandable thanks to
>>> vcld.log file which helped me to backtrace when, how and who called
>> thoose
>>> functions.
>>>
>>> Unfortunately this is not enough because the logic is missing how a
>>> computer will be assigned to an image/reservation. As I saw this
>> allocating
>>> happens in Request.php (fn newReservation()) and maybe the main logic is
>> in
>>> Utils.php (fn allocComputer).
>>
>> You need to look at the isAvailable function in utils.php. That's where a
>> computer is selected for any given reservation.
>>
>>> My problem is that there is no log that I can
>>> use to backtrace what and when happening and I don't understand how this
>>> allocating works.
>>
>> Correct - there is no log of how the computers are allocated. You can use
>> the
>> php function "error_log" to add your own logging to the isAvailable
>> function
>> to see what is happening.
>>
>>> Some questions: Is it random which available computer
>>> will be used? If a computer is preloaded with the same image that a user
>>> chose too then where and how will be taken this into account and filter
>>> that computers which are preloaded with that image?
>>
>> By default, it is not random. Available computers are ordered first by
>> whether or not they are currently loaded with the requested image, then by
>> the
>> specs of the computer with computers having the highest specs ranked first
>> (however, that will be reversed in the next major release so that computers
>> with the lowest specs will be ranked first).
>>
>> There is a config option in the conf.php file that will change the
>> behavior so
>> that computers are randomly assigned rather than being ranked by specs
>> (they
>> are still separated by preloaded/not preloaded).
>>
>>> What happens if a user
>>> requests such an image that is not preloaded on any computer, is it
>> problem
>>> or only the provisioning time period will last a bit more?
>>
>> If a computer is not preloaded, it just makes the provisioning time longer.
>>
>>> Can someone explain the logic behind the allocating procedure and tell me
>>> how the "computer-id" value of reservations table is calculated?
>>
>> See above.
>>
>>> Can you recommend me some technique to trace the "happenings" in php
>>> scripts? I tried to use Zend Server instead of php5 server but there is
>>> something wrong. Have you tried to use it with Zend Server, any
>>> documentation or it shouldn't be problem?
>>
>> I've never used Zend Server. I'd recommend using the error_log function as
>> described above.
>>
>>> Thank you in advance!
>>
>> You're welcome.
>>
>>> Best regards,
>>> Norbert
>>
>> Josh
>>
>>> My advisor is Imre Kocsis and he suggested to mail here:
>>> "Hi all,
>>>
>>> I am Norbert's advisor. Norbert, I think you should have posted this to
>> the
>>> 'dev' list rather than 'user'. Anyways, back in the summer when we
>> started
>>> getting our feet wet with VCL I managed to find only a short and
>> incomplete
>>> thread on the mailing list explaining the (largely undocumented) machine
>>> allocation and preparation scheduling logic - so any help is greatly
>>> appreciated. (Although based on our experience with the code base so far
>> I
>>> think the necessary information can be traced back from the source with
>>> manageable effort.)
>>>
>>> To offer a bit of context to Norbert's questions: the larger goal is to
>>> have a simple discrete event simulator for a hybrid VCL setup in order to
>>> be able to play around with various reservation arrival processes, cost
>> and
>>> possibly fault models. (I know that 'hybrid' VCL setups - as in mixing an
>>> in-house data center e.g. with EC2 - is not a technical reality for VCL
>>> yet, but we are very interested in how it could work from the long-term
>>> cost optimization and risk mitigation point of view for 'hardware budget
>>> conscious' VCL setups.)
>>>
>>> After understanding the necessary parts of VCL, hopefully we will be able
>>> to contribute some sort of documentation on (our understanding of) the
>>> internals back to the community.
>>>
>>> Best regards
>>> Imre"
>> --
>> -------------------------------
>> Josh Thompson
>> Systems Programmer
>> Advanced Computing | VCL Developer
>> North Carolina State University
>>
>> Josh_Thompson@ncsu.edu
>> 919-515-5323
>>
>> my GPG/PGP key can be found at pgp.mit.edu
>>
>> All electronic mail messages in connection with State business which
>> are sent to or received by this account are subject to the NC Public
>> Records Law and may be disclosed to third parties.
Re: Logic of allocating computer to reservation + VCL on Zend Server
Posted by "Madarasz, Norbert" <n....@gmail.com>.
Hi Josh,
first of all thank you for your help, it helped a lot. Now I can say I
mostly understand the procedure with help of error_log.
Further question1: In fn isAvailable there is a part to find computers
whose hosts can handle the required RAM. Would not be necessary to have
something same for CPU?
FQ2: You mentioned ranking will be reversed (say Order by Asc in SQL query
part of
#image.OSid->OS.installtype->OSinstalltype.id->provisioningOSinstalltype.provisioningid->computer.provisioningid)
in the next major release so that computers
with the lowest specs will be ranked first. Is it better to keep the
computers with harder specs for future requests?
Thanks in advance,
Norbert
On 15 April 2013 15:01, Josh Thompson <jo...@ncsu.edu> wrote:
> Norbert,
>
> Answers are inline.
>
> On Monday, April 15, 2013 8:29:34 AM Madarasz, Norbert wrote:
> > Hi all,
> >
> > This is the first time to write you. As part of my master's thesis at
> > Budapest University I should simulate the key functions of VCL with
> > CloudSim.
> >
> > For that the allocating logic is very important part means how a computer
> > is assigned to a reservation. I checked the cwiki page of VCL but the
> exact
> > procedure is not published. I've started to look inside the code and
> found
> > the predictive levels in PERL modules and it was understandable thanks to
> > vcld.log file which helped me to backtrace when, how and who called
> thoose
> > functions.
> >
> > Unfortunately this is not enough because the logic is missing how a
> > computer will be assigned to an image/reservation. As I saw this
> allocating
> > happens in Request.php (fn newReservation()) and maybe the main logic is
> in
> > Utils.php (fn allocComputer).
>
> You need to look at the isAvailable function in utils.php. That's where a
> computer is selected for any given reservation.
>
> > My problem is that there is no log that I can
> > use to backtrace what and when happening and I don't understand how this
> > allocating works.
>
> Correct - there is no log of how the computers are allocated. You can use
> the
> php function "error_log" to add your own logging to the isAvailable
> function
> to see what is happening.
>
> > Some questions: Is it random which available computer
> > will be used? If a computer is preloaded with the same image that a user
> > chose too then where and how will be taken this into account and filter
> > that computers which are preloaded with that image?
>
> By default, it is not random. Available computers are ordered first by
> whether or not they are currently loaded with the requested image, then by
> the
> specs of the computer with computers having the highest specs ranked first
> (however, that will be reversed in the next major release so that computers
> with the lowest specs will be ranked first).
>
> There is a config option in the conf.php file that will change the
> behavior so
> that computers are randomly assigned rather than being ranked by specs
> (they
> are still separated by preloaded/not preloaded).
>
> > What happens if a user
> > requests such an image that is not preloaded on any computer, is it
> problem
> > or only the provisioning time period will last a bit more?
>
> If a computer is not preloaded, it just makes the provisioning time longer.
>
> > Can someone explain the logic behind the allocating procedure and tell me
> > how the "computer-id" value of reservations table is calculated?
>
> See above.
>
> > Can you recommend me some technique to trace the "happenings" in php
> > scripts? I tried to use Zend Server instead of php5 server but there is
> > something wrong. Have you tried to use it with Zend Server, any
> > documentation or it shouldn't be problem?
>
> I've never used Zend Server. I'd recommend using the error_log function as
> described above.
>
> > Thank you in advance!
>
> You're welcome.
>
> > Best regards,
> > Norbert
>
> Josh
>
> > My advisor is Imre Kocsis and he suggested to mail here:
> > "Hi all,
> >
> > I am Norbert's advisor. Norbert, I think you should have posted this to
> the
> > 'dev' list rather than 'user'. Anyways, back in the summer when we
> started
> > getting our feet wet with VCL I managed to find only a short and
> incomplete
> > thread on the mailing list explaining the (largely undocumented) machine
> > allocation and preparation scheduling logic - so any help is greatly
> > appreciated. (Although based on our experience with the code base so far
> I
> > think the necessary information can be traced back from the source with
> > manageable effort.)
> >
> > To offer a bit of context to Norbert's questions: the larger goal is to
> > have a simple discrete event simulator for a hybrid VCL setup in order to
> > be able to play around with various reservation arrival processes, cost
> and
> > possibly fault models. (I know that 'hybrid' VCL setups - as in mixing an
> > in-house data center e.g. with EC2 - is not a technical reality for VCL
> > yet, but we are very interested in how it could work from the long-term
> > cost optimization and risk mitigation point of view for 'hardware budget
> > conscious' VCL setups.)
> >
> > After understanding the necessary parts of VCL, hopefully we will be able
> > to contribute some sort of documentation on (our understanding of) the
> > internals back to the community.
> >
> > Best regards
> > Imre"
> --
> -------------------------------
> Josh Thompson
> Systems Programmer
> Advanced Computing | VCL Developer
> North Carolina State University
>
> Josh_Thompson@ncsu.edu
> 919-515-5323
>
> my GPG/PGP key can be found at pgp.mit.edu
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
Re: Logic of allocating computer to reservation + VCL on Zend Server
Posted by Josh Thompson <jo...@ncsu.edu>.
Norbert,
Answers are inline.
On Monday, April 15, 2013 8:29:34 AM Madarasz, Norbert wrote:
> Hi all,
>
> This is the first time to write you. As part of my master's thesis at
> Budapest University I should simulate the key functions of VCL with
> CloudSim.
>
> For that the allocating logic is very important part means how a computer
> is assigned to a reservation. I checked the cwiki page of VCL but the exact
> procedure is not published. I've started to look inside the code and found
> the predictive levels in PERL modules and it was understandable thanks to
> vcld.log file which helped me to backtrace when, how and who called thoose
> functions.
>
> Unfortunately this is not enough because the logic is missing how a
> computer will be assigned to an image/reservation. As I saw this allocating
> happens in Request.php (fn newReservation()) and maybe the main logic is in
> Utils.php (fn allocComputer).
You need to look at the isAvailable function in utils.php. That's where a
computer is selected for any given reservation.
> My problem is that there is no log that I can
> use to backtrace what and when happening and I don't understand how this
> allocating works.
Correct - there is no log of how the computers are allocated. You can use the
php function "error_log" to add your own logging to the isAvailable function
to see what is happening.
> Some questions: Is it random which available computer
> will be used? If a computer is preloaded with the same image that a user
> chose too then where and how will be taken this into account and filter
> that computers which are preloaded with that image?
By default, it is not random. Available computers are ordered first by
whether or not they are currently loaded with the requested image, then by the
specs of the computer with computers having the highest specs ranked first
(however, that will be reversed in the next major release so that computers
with the lowest specs will be ranked first).
There is a config option in the conf.php file that will change the behavior so
that computers are randomly assigned rather than being ranked by specs (they
are still separated by preloaded/not preloaded).
> What happens if a user
> requests such an image that is not preloaded on any computer, is it problem
> or only the provisioning time period will last a bit more?
If a computer is not preloaded, it just makes the provisioning time longer.
> Can someone explain the logic behind the allocating procedure and tell me
> how the "computer-id" value of reservations table is calculated?
See above.
> Can you recommend me some technique to trace the "happenings" in php
> scripts? I tried to use Zend Server instead of php5 server but there is
> something wrong. Have you tried to use it with Zend Server, any
> documentation or it shouldn't be problem?
I've never used Zend Server. I'd recommend using the error_log function as
described above.
> Thank you in advance!
You're welcome.
> Best regards,
> Norbert
Josh
> My advisor is Imre Kocsis and he suggested to mail here:
> "Hi all,
>
> I am Norbert's advisor. Norbert, I think you should have posted this to the
> 'dev' list rather than 'user'. Anyways, back in the summer when we started
> getting our feet wet with VCL I managed to find only a short and incomplete
> thread on the mailing list explaining the (largely undocumented) machine
> allocation and preparation scheduling logic - so any help is greatly
> appreciated. (Although based on our experience with the code base so far I
> think the necessary information can be traced back from the source with
> manageable effort.)
>
> To offer a bit of context to Norbert's questions: the larger goal is to
> have a simple discrete event simulator for a hybrid VCL setup in order to
> be able to play around with various reservation arrival processes, cost and
> possibly fault models. (I know that 'hybrid' VCL setups - as in mixing an
> in-house data center e.g. with EC2 - is not a technical reality for VCL
> yet, but we are very interested in how it could work from the long-term
> cost optimization and risk mitigation point of view for 'hardware budget
> conscious' VCL setups.)
>
> After understanding the necessary parts of VCL, hopefully we will be able
> to contribute some sort of documentation on (our understanding of) the
> internals back to the community.
>
> Best regards
> Imre"
--
-------------------------------
Josh Thompson
Systems Programmer
Advanced Computing | VCL Developer
North Carolina State University
Josh_Thompson@ncsu.edu
919-515-5323
my GPG/PGP key can be found at pgp.mit.edu
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.