You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@vcl.apache.org by Khder Omar <kh...@gmail.com> on 2014/10/21 16:39:05 UTC
Scheduling in VCL
Dear all,
We were wondering what kind of scheduler algorithms VCL might use ? Any
hints about the scheduler source code will be appreciated. The idea is
actually to check how VCL will perform while changing the scheduler
algorithm [in management node].
Thanks in advance
Best regards,
Phd team
Re: Scheduling in VCL
Posted by Josh Thompson <jo...@ncsu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Khder,
The biggest thing missing is selection of a VM based on load on the
hypervisors. However, there is not currently a way for the frontend to know
what the load on the hypervisors is to be able to use that information for
selection of a VM. So, to add this, the backend would need to collect
hypervisor load information periodically and push that to the database so that
the frontend could analyze it in selecting VMs.
Josh
On Friday, November 07, 2014 11:08:08 PM Khder Omar wrote:
> Josh,
> Thanks for the detailed explanation ! I really appreciate such sharing
> information! Eventually, I found important to see how such internal process
> works in order to know better what to improve and experiment. Based on your
> statement, I can see that it misses the limit based on number of cores.
> In the other hand, do you think that still some few things missed or not
> being well performed in the scheduling of the VCL for example ? or queuing
> system ?
>
> Best Regards,
> Phd team
>
> 2014-11-07 22:40 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Khder,
> >
> > VCL knows about the specs (RAM, cores, processor speed) of each VM host
> > and
> > the specs of each image. It does not assign a reservation to a VM on a
> > host
> > that would result in the host's RAM being overbooked. The number of cores
> > are
> > not currently checked. So, they could be overbooked. Functionality to
> > limit
> > reservations based on number of cores will be added at a future date. So,
> > yes, it is the responsibility of the hypervisor to assign the required
> > resources, but it should never have to overbook the RAM to bring up a VM
> > for a
> > new reservation. For bare metal reservations, an image is never assigned
> > to a
> > node that doesn't meet the minimum specs of the image.
> >
> > Multiple users can make reservations for the same image. A separate node
> > is
> > deployed for every reservation. There could be a slight impact on deploy
> > time
> > since VCL uses linked clones for the disk files. So, reads for the same
> > image
> > would be hitting the same file. Write would be going to separate files.
> > This
> > is actually somewhat complicated because the base file on which the linked
> > clones is actually only shared with hosts that are using the same
> > datastore.
> >
> > When a node is already loaded with an image someone requests (we call this
> > a
> > hot standby node), the deploy time is obviously much shorter. VCL does
> > not
> > allow nodes to be shared other than the case of a bare metal machine
> > running a
> > hypervisor and hosting VMs (the VMs aren't shared, but the physical
> > machine
> > is). So, only one user would get a given preloaded node.
> >
> > Josh
> >
> > On Thursday, November 06, 2014 2:46:59 AM Khder Omar wrote:
> > > Dear Josh,
> > >
> > > Thanks again for your reply ! I understand the mapping part ! Concerning
> > > the VCL model as you might mentioned the terms could be different
> >
> > comparing
> >
> > > to a typical IaaS.
> > > Most probably I was bit confusing the images provisioning. If I
> >
> > understood
> >
> > > it correctly, once reservation process starts, a new image will be
> >
> > deployed
> >
> > > in the corresponding computer or host [just streaming the disk images
> >
> > files
> >
> > > to build a complete environment within specific resources CPU and RAM]
> > >
> > > Therefore, that will be the Physical/Virtual deployer [Hypervisor] role
> >
> > to
> >
> > > assign those resources ?
> > >
> > > Another interesting point which highlights the point if many users can
> >
> > make
> >
> > > more than one reservation for the same image ? does it affect the image
> > > loading time [queue of requests to access/use shared resource] ?
> > > besides,
> > > if the image is already deployed and running in a specific computer [can
> >
> > be
> >
> > > physical/virtual], can many users access the same computer at the same
> >
> > time
> >
> > > ?
> > >
> > > Thanks for your clarification !
> > >
> > > Best regards,
> > > Phd Team
> > >
> > > 2014-11-05 22:34 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > > > -----BEGIN PGP SIGNED MESSAGE-----
> > > > Hash: SHA1
> > > >
> > > > Khder,
> > > >
> > > > Mapping is just a way for the scheduler part of the frontend to know
> >
> > what
> >
> > > > computers a given image can be loaded on.This could be because of an
> > > > architectural restriction or it could simply be because a researcher
> > > > bought
> > > > her own hardware and only her images should be run on them. Mapping
> > > > doesn't
> > > > actually have to do with where the image files reside. I don't think
> >
> > the
> >
> > > > backend code ever even looks at the mapping information.
> > > >
> > > > Remember that VCL predates the terms cloud computing, IaaS, PaaS, etc.
> > > > So,
> > > > the terms used for VCL might be somewhat different from more recent
> > > > systems.
> > > >
> > > > In VCL, each "image" or "environment" consists of a set of disk image
> > > > files
> > > > and the associated information about the image that is in the
> > > > database.
> > > > Initial "base images" must be created somewhat manually. Then,
> > > > further
> > > > images
> > > > are derived from those base images. I don't know that VCL really has
> > > > flavors
> > > > because each image will have its own CPU and RAM specifications.
> > > >
> > > > Does that answer your questions?
> > > >
> > > > Josh
> > > >
> > > > On Tuesday, November 04, 2014 11:00:01 PM Khder Omar wrote:
> > > > > Dear Josh,
> > > > >
> > > > > I was wondering during the creation of new image, it does not
> >
> > require to
> >
> > > > be
> > > >
> > > > > stored in the computer node itself, it can be stored in an image
> >
> > library
> >
> > > > > like a SAN or NAS storage then we mapped a given image to a certain
> > > > > computer or computer group ! the fact to 'map' an image or image
> >
> > group
> >
> > > > to a
> > > >
> > > > > computer or computer group, does it mean 'streaming' the image
> >
> > 'locally'
> >
> > > > to
> > > >
> > > > > the computer node ? or only during the reservation ?
> > > > > I was bit confused between the terms used in VCL and in a typical
> >
> > IaaS
> >
> > > > > environment: the image is claimed as a template of a virtual
> >
> > machine, in
> >
> > > > > VCL, I have noticed that the image is being created from a base OS
> >
> > and
> >
> > > > > resources information are assigned in the database such as RAM, CPU,
> > > >
> > > > arch,
> > > >
> > > > > id etc...
> > > > > Then I might assume [correct me if I understood it wrong] the
> >
> > Virtual
> >
> > > > > Machine will be initiated [considering an Hypervisor KVM or ESXi ]
> >
> > while
> >
> > > > > processing the reservation. Therefore, we can describe such image in
> >
> > a
> >
> > > > > typical IaaS as a virtual machine flavor ready to be deployed and
> >
> > run on
> >
> > > > > Host or Computer [within an hypervisor supported].
> > > > >
> > > > > Thanks!
> > > > >
> > > > > PhD team
> > > > >
> > > > > 2014-10-30 20:07 GMT+01:00 Khder Omar <kh...@gmail.com>:
> > > > > > Dear Josh,
> > > > > >
> > > > > > That is true ! Thanks again for the clarification ! I will start
> > > >
> > > > doing
> > > >
> > > > > > some experiments and see what it gives from performance
> >
> > perspective !
> >
> > > > In
> > > >
> > > > > > the other hand, I will keep updating for any proposal of ideas can
> >
> > be
> >
> > > > > > generated then we can refine the scheduler code and retest again !
> > > > > >
> > > > > > Many thanks for sharing such valuable information!
> > > > > >
> > > > > > We will keep updated !
> > > > > >
> > > > > > Best Regards
> > > > > > PhD Team
> > > > > >
> > > > > > 2014-10-30 19:56 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > > > > >> -----BEGIN PGP SIGNED MESSAGE-----
> > > > > >> Hash: SHA1
> > > > > >>
> > > > > >> Khder,
> > > > > >>
> > > > > >> The scheduler only allocates a computer randomly if
> > > > > >> SCHEDULER_ALLOCATE_RANDOM_COMPUTER is set to 1 in conf.php, which
> >
> > it
> >
> > > > is
> > > >
> > > > > >> not by
> > > > > >> default.
> > > > > >>
> > > > > >> Yes, you could create pools that each have a weight or ranking
> > > >
> > > > assigned.
> > > >
> > > > > >> Each
> > > > > >> computer would be assigned to a pool. Then, when the initial set
> >
> > of
> >
> > > > > >> computers
> > > > > >> is generated, they could be ranked by the pool weight/ranks. It
> > > > > >> would
> > > > > >> then be
> > > > > >> up to an administer or external application to manage the
> >
> > weight/rank
> >
> > > > of
> > > >
> > > > > >> each
> > > > > >> pool.
> > > > > >>
> > > > > >> Josh
> > > > > >>
> > > > > >> On Thursday, October 30, 2014 7:48:50 PM Khder Omar wrote:
> > > > > >> > Dear Josh,
> > > > > >> >
> > > > > >> > I would like to thank you for the precious hints concerning the
> > > > > >>
> > > > > >> scheduling
> > > > > >>
> > > > > >> > nature within VCL. I can see that the set of computers is
> >
> > filtered
> >
> > > > in
> > > >
> > > > > >> > advance and then the scheduler will pick up RANDOMLY one of
> >
> > them.
> >
> > > > > >> > To
> > > > > >>
> > > > > >> check
> > > > > >>
> > > > > >> > the possibilities to extend the function considering a large
> >
> > scale
> >
> > > > > >> > environment where we can introduce the 'region' or 'pool' term
> >
> > in
> >
> > > > the
> > > >
> > > > > >> code.
> > > > > >>
> > > > > >> > With multiple management nodes, do you think that using the
> > > > > >> > 'weight'
> > > > > >> > for
> > > > > >> > the workload per pool and then per computer therefore will rank
> >
> > it
> >
> > > > will
> > > >
> > > > > >> be
> > > > > >>
> > > > > >> > more precise on how to balance the load across the hole
> > > >
> > > > infrastructure
> > > >
> > > > > >> > ?
> > > > > >> >
> > > > > >> > Best Regards,
> > > > > >> > PhD team
> > > > > >> >
> > > > > >> > 2014-10-30 19:28 GMT+01:00 Josh Thompson <
> >
> > josh_thompson@ncsu.edu>:
> > > > > >> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > > > >> > > Hash: SHA1
> > > > > >> > >
> > > > > >> > > Khder,
> > > > > >> > >
> > > > > >> > > Sorry for the late response.
> > > > > >> > >
> > > > > >> > > You can certainly experiment with extending the code to
> >
> > include
> >
> > > > more
> > > >
> > > > > >> > > advanced
> > > > > >> > > scheduling functions. The existing ranking of computers is
> > > >
> > > > something
> > > >
> > > > > >> like
> > > > > >>
> > > > > >> > > this:
> > > > > >> > >
> > > > > >> > > 1) generate a set of all computers meeting the minimum
> > > >
> > > > requirements
> > > >
> > > > > >> of the
> > > > > >>
> > > > > >> > > image, that the user has access to, and that the image is
> >
> > mapped
> >
> > > > to;
> > > >
> > > > > >> order
> > > > > >>
> > > > > >> > > that list by node specs from low to high
> > > > > >> > > 2) remove from that set any computers that are already
> >
> > assigned
> >
> > > > > >> > > reservations
> > > > > >> > > 3) if the image is virtual, remove any VMs for which the host
> > > >
> > > > doesn't
> > > >
> > > > > >> have
> > > > > >>
> > > > > >> > > enough memory to load the VM (without overbooking)
> > > > > >> > > 4) assign the new reservation the first computer from that
> > > > > >> > > set
> > > > > >> > > for
> > > > > >>
> > > > > >> which
> > > > > >>
> > > > > >> > > an
> > > > > >> > > active management node can be found
> > > > > >> > >
> > > > > >> > > There's actually 3 sets of computers being tracked - those
> > > > > >> > > already
> > > > > >>
> > > > > >> loaded
> > > > > >>
> > > > > >> > > with
> > > > > >> > > the image being requested, those set aside for a block
> >
> > allocation
> >
> > > > of
> > > >
> > > > > >> which
> > > > > >>
> > > > > >> > > the
> > > > > >> > > user is a member, and then a set of both of those plus any
> > > >
> > > > available
> > > >
> > > > > >> > > computers
> > > > > >> > > that are currently loaded with another image.
> > > > > >> > >
> > > > > >> > > There is a configuration option
> > > >
> > > > (SCHEDULER_ALLOCATE_RANDOM_COMPUTER)
> > > >
> > > > > >> in
> > > > > >>
> > > > > >> > > conf.php that allows the sets of computers to be randomized.
> >
> > The
> >
> > > > > >> > > motivation
> > > > > >> > > behind this option is for sites that have homogeneous virtual
> > > > > >>
> > > > > >> machines and
> > > > > >>
> > > > > >> > > hosts. Randomizing the assigned computer should help spread
> >
> > the
> >
> > > > load
> > > >
> > > > > >> > > equally
> > > > > >> > > among all existing hosts.
> > > > > >> > >
> > > > > >> > > I hope that helps.
> > > > > >> > >
> > > > > >> > > Josh
> > > > > >> > >
> > > > > >> > > On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote:
> > > > > >> > > > Hi Josh,
> > > > > >> > > >
> > > > > >> > > > Thanks for your reaction. I have checked the code source
> >
> > and I
> >
> > > > > >> think the
> > > > > >>
> > > > > >> > > > only function which might fulfill my question is function
> > > > > >> > >
> > > > > >> > > allocComputer(...)
> > > > > >> > >
> > > > > >> > > > Eventually, the function a piece test schedule code
> > > > > >> > > >
> > > > > >> > > > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) {
> > > > > >> > > >
> > > > > >> > > > shuffle($blockids);
> > > > > >> > > > shuffle($currentids);
> > > > > >> > > > shuffle($computerids);
> > > > > >> > > >
> > > > > >> > > > }
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > > I might assume if it was correctly understood that VCL
> > > > > >> > > > scheduler
> > > > > >>
> > > > > >> process
> > > > > >>
> > > > > >> > > > determines in first place a computer to be assigned to a
> > > >
> > > > management
> > > >
> > > > > >> node
> > > > > >>
> > > > > >> > > > from a given array or table in order otherwise it can be
> > > >
> > > > randomly
> > > >
> > > > > >> > > > chosen!
> > > > > >> > > > Then, we can assume that the scheduling decision is still
> >
> > using
> >
> > > > a
> > > >
> > > > > >> basic
> > > > > >>
> > > > > >> > > > order as FCFS or a chance/Random order! It that assumed be
> > > >
> > > > correct
> > > >
> > > > > >> ? if
> > > > > >>
> > > > > >> > > so,
> > > > > >> > >
> > > > > >> > > > is there a way to extend the code by including more
> > > > > >> > > > advanced
> > > > > >>
> > > > > >> scheduling
> > > > > >>
> > > > > >> > > > functions ?
> > > > > >> > > >
> > > > > >> > > > Best regards,
> > > > > >> > > > PhD Team
> > > > > >> > > >
> > > > > >> > > > 2014-10-21 16:11 GMT+01:00 Josh Thompson <
> > > >
> > > > josh_thompson@ncsu.edu>:
> > > > > >> > > > > -----BEGIN PGP SIGNED MESSAGE-----
> > > > > >> > > > > Hash: SHA1
> > > > > >> > > > >
> > > > > >> > > > > Khder,
> > > > > >> > > > >
> > > > > >> > > > > The scheduling of reservations is actually done in the
> > > > > >> > > > > web
> > > >
> > > > code.
> > > >
> > > > > >> Have
> > > > > >>
> > > > > >> > > a
> > > > > >> > >
> > > > > >> > > > > look
> > > > > >> > > > > at the isAvailable function in the web/.ht-inc/utils.php
> > > > > >> > > > > file.
> > > >
> > > > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
> > > >
> > > > > >> > > > > Josh
> > > > > >> > > > >
> > > > > >> > > > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
> > > > > >> > > > > > Dear all,
> > > > > >> > > > > >
> > > > > >> > > > > > We were wondering what kind of scheduler algorithms VCL
> > > >
> > > > might
> > > >
> > > > > >> use ?
> > > > > >>
> > > > > >> > > Any
> > > > > >> > >
> > > > > >> > > > > > hints about the scheduler source code will be
> >
> > appreciated.
> >
> > > > The
> > > >
> > > > > >> idea
> > > > > >>
> > > > > >> > > is
> > > > > >> > >
> > > > > >> > > > > > actually to check how VCL will perform while changing
> >
> > the
> >
> > > > > >> scheduler
> > > > > >>
> > > > > >> > > > > > algorithm [in management node].
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > Thanks in advance
> > > > > >> > > > > >
> > > > > >> > > > > > Best regards,
> > > > > >> > > > > > Phd team
> > > > > >> > > > >
> > > > > >> > > > > - --
> > > > > >> > > > > - -------------------------------
> > > > > >> > > > > Josh Thompson
> > > > > >> > > > > VCL Developer
> > > > > >> > > > > North Carolina State University
> > > > > >> > > > >
> > > > > >> > > > > my GPG/PGP key can be found at pgp.mit.edu
> > > > > >> > > > >
> > > > > >> > > > > All electronic mail messages in connection with State
> > > > > >> > > > > business
> > > > > >>
> > > > > >> which
> > > > > >>
> > > > > >> > > > > are sent to or received by this account are subject to
> >
> > the NC
> >
> > > > > >> Public
> > > > > >>
> > > > > >> > > > > Records Law and may be disclosed to third parties.
> > > > > >> > > > > -----BEGIN PGP SIGNATURE-----
> > > > > >> > > > > Version: GnuPG v2.0.22 (GNU/Linux)
> > > >
> > > > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
> > > >
> > > > > >> > > > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
> > > > > >> > > > > =RMTU
> > > > > >> > > > > -----END PGP SIGNATURE-----
> > > > > >> > >
> > > > > >> > > - --
> > > > > >> > > - -------------------------------
> > > > > >> > > Josh Thompson
> > > > > >> > > VCL Developer
> > > > > >> > > North Carolina State University
> > > > > >> > >
> > > > > >> > > my GPG/PGP key can be found at pgp.mit.edu
> > > > > >> > >
> > > > > >> > > All electronic mail messages in connection with State
> > > > > >> > > business
> > > >
> > > > which
> > > >
> > > > > >> > > are sent to or received by this account are subject to the NC
> > > >
> > > > Public
> > > >
> > > > > >> > > Records Law and may be disclosed to third parties.
> > > > > >> > > -----BEGIN PGP SIGNATURE-----
> > > > > >> > > Version: GnuPG v2.0.22 (GNU/Linux)
> >
> > iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx
> >
> > > > > >> > > 5uQAniwpvKVjVGdW//rvNBYCKu485Siy
> > > > > >> > > =xWum
> > > > > >> > > -----END PGP SIGNATURE-----
> > > > > >>
> > > > > >> - --
> > > > > >> - -------------------------------
> > > > > >> Josh Thompson
> > > > > >> VCL Developer
> > > > > >> North Carolina State University
> > > > > >>
> > > > > >> my GPG/PGP key can be found at pgp.mit.edu
> > > > > >>
> > > > > >> All electronic mail messages in connection with State business
> >
> > which
> >
> > > > > >> are sent to or received by this account are subject to the NC
> >
> > Public
> >
> > > > > >> Records Law and may be disclosed to third parties.
> > > > > >> -----BEGIN PGP SIGNATURE-----
> > > > > >> Version: GnuPG v2.0.22 (GNU/Linux)
> > > > > >>
> > > > > >> iEYEARECAAYFAlRSieYACgkQV/LQcNdtPQOjlACfS97i9Hk0Tmspy+6vfgOcENht
> > > > > >> UtYAn008A0w5VBq+eg5iGg1GRlff3zDB
> > > > > >> =tCAJ
> > > > > >> -----END PGP SIGNATURE-----
> > > >
> > > > - --
> > > > - -------------------------------
> > > > Josh Thompson
> > > > VCL Developer
> > > > North Carolina State University
> > > >
> > > > my GPG/PGP key can be found at pgp.mit.edu
> > > >
> > > > All electronic mail messages in connection with State business which
> > > > are sent to or received by this account are subject to the NC Public
> > > > Records Law and may be disclosed to third parties.
> > > > -----BEGIN PGP SIGNATURE-----
> > > > Version: GnuPG v2.0.22 (GNU/Linux)
> > > >
> > > > iEYEARECAAYFAlRal+cACgkQV/LQcNdtPQOAGACeL2Sanhs7ShZaCyKcSo5N2BYk
> > > > wLEAn08YbcyiCmR4rY9u4kZJocfxY1Ju
> > > > =pxdL
> > > > -----END PGP SIGNATURE-----
> >
> > - --
> > - -------------------------------
> > Josh Thompson
> > VCL Developer
> > North Carolina State University
> >
> > my GPG/PGP key can be found at pgp.mit.edu
> >
> > All electronic mail messages in connection with State business which
> > are sent to or received by this account are subject to the NC Public
> > Records Law and may be disclosed to third parties.
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v2.0.22 (GNU/Linux)
> >
> > iEYEARECAAYFAlRdPGkACgkQV/LQcNdtPQNFjACfdZm6YIKYhJ3EtamGXOfX1Dgs
> > X90An3OeCGLL58j2VzzA96jTPfBuIDgI
> > =w2De
> > -----END PGP SIGNATURE-----
- --
- -------------------------------
Josh Thompson
VCL Developer
North Carolina State University
my GPG/PGP key can be found at pgp.mit.edu
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iEYEARECAAYFAlRg+moACgkQV/LQcNdtPQO9WACdFe64eIJAA9JCUTHql7kti1Hy
fX4AnjQshbEopN3rdjWEQzZTUrpae9Gr
=kEzB
-----END PGP SIGNATURE-----
Re: Scheduling in VCL
Posted by Khder Omar <kh...@gmail.com>.
Josh,
Thanks for the detailed explanation ! I really appreciate such sharing
information! Eventually, I found important to see how such internal process
works in order to know better what to improve and experiment. Based on your
statement, I can see that it misses the limit based on number of cores.
In the other hand, do you think that still some few things missed or not
being well performed in the scheduling of the VCL for example ? or queuing
system ?
Best Regards,
Phd team
2014-11-07 22:40 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Khder,
>
> VCL knows about the specs (RAM, cores, processor speed) of each VM host and
> the specs of each image. It does not assign a reservation to a VM on a
> host
> that would result in the host's RAM being overbooked. The number of cores
> are
> not currently checked. So, they could be overbooked. Functionality to
> limit
> reservations based on number of cores will be added at a future date. So,
> yes, it is the responsibility of the hypervisor to assign the required
> resources, but it should never have to overbook the RAM to bring up a VM
> for a
> new reservation. For bare metal reservations, an image is never assigned
> to a
> node that doesn't meet the minimum specs of the image.
>
> Multiple users can make reservations for the same image. A separate node
> is
> deployed for every reservation. There could be a slight impact on deploy
> time
> since VCL uses linked clones for the disk files. So, reads for the same
> image
> would be hitting the same file. Write would be going to separate files.
> This
> is actually somewhat complicated because the base file on which the linked
> clones is actually only shared with hosts that are using the same
> datastore.
>
> When a node is already loaded with an image someone requests (we call this
> a
> hot standby node), the deploy time is obviously much shorter. VCL does not
> allow nodes to be shared other than the case of a bare metal machine
> running a
> hypervisor and hosting VMs (the VMs aren't shared, but the physical machine
> is). So, only one user would get a given preloaded node.
>
> Josh
>
> On Thursday, November 06, 2014 2:46:59 AM Khder Omar wrote:
> > Dear Josh,
> >
> > Thanks again for your reply ! I understand the mapping part ! Concerning
> > the VCL model as you might mentioned the terms could be different
> comparing
> > to a typical IaaS.
> > Most probably I was bit confusing the images provisioning. If I
> understood
> > it correctly, once reservation process starts, a new image will be
> deployed
> > in the corresponding computer or host [just streaming the disk images
> files
> > to build a complete environment within specific resources CPU and RAM]
> >
> > Therefore, that will be the Physical/Virtual deployer [Hypervisor] role
> to
> > assign those resources ?
> >
> > Another interesting point which highlights the point if many users can
> make
> > more than one reservation for the same image ? does it affect the image
> > loading time [queue of requests to access/use shared resource] ? besides,
> > if the image is already deployed and running in a specific computer [can
> be
> > physical/virtual], can many users access the same computer at the same
> time
> > ?
> >
> > Thanks for your clarification !
> >
> > Best regards,
> > Phd Team
> >
> > 2014-11-05 22:34 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA1
> > >
> > > Khder,
> > >
> > > Mapping is just a way for the scheduler part of the frontend to know
> what
> > > computers a given image can be loaded on.This could be because of an
> > > architectural restriction or it could simply be because a researcher
> > > bought
> > > her own hardware and only her images should be run on them. Mapping
> > > doesn't
> > > actually have to do with where the image files reside. I don't think
> the
> > > backend code ever even looks at the mapping information.
> > >
> > > Remember that VCL predates the terms cloud computing, IaaS, PaaS, etc.
> > > So,
> > > the terms used for VCL might be somewhat different from more recent
> > > systems.
> > >
> > > In VCL, each "image" or "environment" consists of a set of disk image
> > > files
> > > and the associated information about the image that is in the database.
> > > Initial "base images" must be created somewhat manually. Then, further
> > > images
> > > are derived from those base images. I don't know that VCL really has
> > > flavors
> > > because each image will have its own CPU and RAM specifications.
> > >
> > > Does that answer your questions?
> > >
> > > Josh
> > >
> > > On Tuesday, November 04, 2014 11:00:01 PM Khder Omar wrote:
> > > > Dear Josh,
> > > >
> > > > I was wondering during the creation of new image, it does not
> require to
> > >
> > > be
> > >
> > > > stored in the computer node itself, it can be stored in an image
> library
> > > > like a SAN or NAS storage then we mapped a given image to a certain
> > > > computer or computer group ! the fact to 'map' an image or image
> group
> > >
> > > to a
> > >
> > > > computer or computer group, does it mean 'streaming' the image
> 'locally'
> > >
> > > to
> > >
> > > > the computer node ? or only during the reservation ?
> > > > I was bit confused between the terms used in VCL and in a typical
> IaaS
> > > > environment: the image is claimed as a template of a virtual
> machine, in
> > > > VCL, I have noticed that the image is being created from a base OS
> and
> > > > resources information are assigned in the database such as RAM, CPU,
> > >
> > > arch,
> > >
> > > > id etc...
> > > > Then I might assume [correct me if I understood it wrong] the
> Virtual
> > > > Machine will be initiated [considering an Hypervisor KVM or ESXi ]
> while
> > > > processing the reservation. Therefore, we can describe such image in
> a
> > > > typical IaaS as a virtual machine flavor ready to be deployed and
> run on
> > > > Host or Computer [within an hypervisor supported].
> > > >
> > > > Thanks!
> > > >
> > > > PhD team
> > > >
> > > > 2014-10-30 20:07 GMT+01:00 Khder Omar <kh...@gmail.com>:
> > > > > Dear Josh,
> > > > >
> > > > > That is true ! Thanks again for the clarification ! I will start
> > >
> > > doing
> > >
> > > > > some experiments and see what it gives from performance
> perspective !
> > >
> > > In
> > >
> > > > > the other hand, I will keep updating for any proposal of ideas can
> be
> > > > > generated then we can refine the scheduler code and retest again !
> > > > >
> > > > > Many thanks for sharing such valuable information!
> > > > >
> > > > > We will keep updated !
> > > > >
> > > > > Best Regards
> > > > > PhD Team
> > > > >
> > > > > 2014-10-30 19:56 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > > > >> -----BEGIN PGP SIGNED MESSAGE-----
> > > > >> Hash: SHA1
> > > > >>
> > > > >> Khder,
> > > > >>
> > > > >> The scheduler only allocates a computer randomly if
> > > > >> SCHEDULER_ALLOCATE_RANDOM_COMPUTER is set to 1 in conf.php, which
> it
> > >
> > > is
> > >
> > > > >> not by
> > > > >> default.
> > > > >>
> > > > >> Yes, you could create pools that each have a weight or ranking
> > >
> > > assigned.
> > >
> > > > >> Each
> > > > >> computer would be assigned to a pool. Then, when the initial set
> of
> > > > >> computers
> > > > >> is generated, they could be ranked by the pool weight/ranks. It
> > > > >> would
> > > > >> then be
> > > > >> up to an administer or external application to manage the
> weight/rank
> > >
> > > of
> > >
> > > > >> each
> > > > >> pool.
> > > > >>
> > > > >> Josh
> > > > >>
> > > > >> On Thursday, October 30, 2014 7:48:50 PM Khder Omar wrote:
> > > > >> > Dear Josh,
> > > > >> >
> > > > >> > I would like to thank you for the precious hints concerning the
> > > > >>
> > > > >> scheduling
> > > > >>
> > > > >> > nature within VCL. I can see that the set of computers is
> filtered
> > >
> > > in
> > >
> > > > >> > advance and then the scheduler will pick up RANDOMLY one of
> them.
> > > > >> > To
> > > > >>
> > > > >> check
> > > > >>
> > > > >> > the possibilities to extend the function considering a large
> scale
> > > > >> > environment where we can introduce the 'region' or 'pool' term
> in
> > >
> > > the
> > >
> > > > >> code.
> > > > >>
> > > > >> > With multiple management nodes, do you think that using the
> > > > >> > 'weight'
> > > > >> > for
> > > > >> > the workload per pool and then per computer therefore will rank
> it
> > >
> > > will
> > >
> > > > >> be
> > > > >>
> > > > >> > more precise on how to balance the load across the hole
> > >
> > > infrastructure
> > >
> > > > >> > ?
> > > > >> >
> > > > >> > Best Regards,
> > > > >> > PhD team
> > > > >> >
> > > > >> > 2014-10-30 19:28 GMT+01:00 Josh Thompson <
> josh_thompson@ncsu.edu>:
> > > > >> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > > >> > > Hash: SHA1
> > > > >> > >
> > > > >> > > Khder,
> > > > >> > >
> > > > >> > > Sorry for the late response.
> > > > >> > >
> > > > >> > > You can certainly experiment with extending the code to
> include
> > >
> > > more
> > >
> > > > >> > > advanced
> > > > >> > > scheduling functions. The existing ranking of computers is
> > >
> > > something
> > >
> > > > >> like
> > > > >>
> > > > >> > > this:
> > > > >> > >
> > > > >> > > 1) generate a set of all computers meeting the minimum
> > >
> > > requirements
> > >
> > > > >> of the
> > > > >>
> > > > >> > > image, that the user has access to, and that the image is
> mapped
> > >
> > > to;
> > >
> > > > >> order
> > > > >>
> > > > >> > > that list by node specs from low to high
> > > > >> > > 2) remove from that set any computers that are already
> assigned
> > > > >> > > reservations
> > > > >> > > 3) if the image is virtual, remove any VMs for which the host
> > >
> > > doesn't
> > >
> > > > >> have
> > > > >>
> > > > >> > > enough memory to load the VM (without overbooking)
> > > > >> > > 4) assign the new reservation the first computer from that set
> > > > >> > > for
> > > > >>
> > > > >> which
> > > > >>
> > > > >> > > an
> > > > >> > > active management node can be found
> > > > >> > >
> > > > >> > > There's actually 3 sets of computers being tracked - those
> > > > >> > > already
> > > > >>
> > > > >> loaded
> > > > >>
> > > > >> > > with
> > > > >> > > the image being requested, those set aside for a block
> allocation
> > >
> > > of
> > >
> > > > >> which
> > > > >>
> > > > >> > > the
> > > > >> > > user is a member, and then a set of both of those plus any
> > >
> > > available
> > >
> > > > >> > > computers
> > > > >> > > that are currently loaded with another image.
> > > > >> > >
> > > > >> > > There is a configuration option
> > >
> > > (SCHEDULER_ALLOCATE_RANDOM_COMPUTER)
> > >
> > > > >> in
> > > > >>
> > > > >> > > conf.php that allows the sets of computers to be randomized.
> The
> > > > >> > > motivation
> > > > >> > > behind this option is for sites that have homogeneous virtual
> > > > >>
> > > > >> machines and
> > > > >>
> > > > >> > > hosts. Randomizing the assigned computer should help spread
> the
> > >
> > > load
> > >
> > > > >> > > equally
> > > > >> > > among all existing hosts.
> > > > >> > >
> > > > >> > > I hope that helps.
> > > > >> > >
> > > > >> > > Josh
> > > > >> > >
> > > > >> > > On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote:
> > > > >> > > > Hi Josh,
> > > > >> > > >
> > > > >> > > > Thanks for your reaction. I have checked the code source
> and I
> > > > >>
> > > > >> think the
> > > > >>
> > > > >> > > > only function which might fulfill my question is function
> > > > >> > >
> > > > >> > > allocComputer(...)
> > > > >> > >
> > > > >> > > > Eventually, the function a piece test schedule code
> > > > >> > > >
> > > > >> > > > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) {
> > > > >> > > >
> > > > >> > > > shuffle($blockids);
> > > > >> > > > shuffle($currentids);
> > > > >> > > > shuffle($computerids);
> > > > >> > > >
> > > > >> > > > }
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > I might assume if it was correctly understood that VCL
> > > > >> > > > scheduler
> > > > >>
> > > > >> process
> > > > >>
> > > > >> > > > determines in first place a computer to be assigned to a
> > >
> > > management
> > >
> > > > >> node
> > > > >>
> > > > >> > > > from a given array or table in order otherwise it can be
> > >
> > > randomly
> > >
> > > > >> > > > chosen!
> > > > >> > > > Then, we can assume that the scheduling decision is still
> using
> > >
> > > a
> > >
> > > > >> basic
> > > > >>
> > > > >> > > > order as FCFS or a chance/Random order! It that assumed be
> > >
> > > correct
> > >
> > > > >> ? if
> > > > >>
> > > > >> > > so,
> > > > >> > >
> > > > >> > > > is there a way to extend the code by including more advanced
> > > > >>
> > > > >> scheduling
> > > > >>
> > > > >> > > > functions ?
> > > > >> > > >
> > > > >> > > > Best regards,
> > > > >> > > > PhD Team
> > > > >> > > >
> > > > >> > > > 2014-10-21 16:11 GMT+01:00 Josh Thompson <
> > >
> > > josh_thompson@ncsu.edu>:
> > > > >> > > > > -----BEGIN PGP SIGNED MESSAGE-----
> > > > >> > > > > Hash: SHA1
> > > > >> > > > >
> > > > >> > > > > Khder,
> > > > >> > > > >
> > > > >> > > > > The scheduling of reservations is actually done in the web
> > >
> > > code.
> > >
> > > > >> Have
> > > > >>
> > > > >> > > a
> > > > >> > >
> > > > >> > > > > look
> > > > >> > > > > at the isAvailable function in the web/.ht-inc/utils.php
> > > > >> > > > > file.
> > >
> > > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
> > >
> > > > >> > > > > Josh
> > > > >> > > > >
> > > > >> > > > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
> > > > >> > > > > > Dear all,
> > > > >> > > > > >
> > > > >> > > > > > We were wondering what kind of scheduler algorithms VCL
> > >
> > > might
> > >
> > > > >> use ?
> > > > >>
> > > > >> > > Any
> > > > >> > >
> > > > >> > > > > > hints about the scheduler source code will be
> appreciated.
> > >
> > > The
> > >
> > > > >> idea
> > > > >>
> > > > >> > > is
> > > > >> > >
> > > > >> > > > > > actually to check how VCL will perform while changing
> the
> > > > >>
> > > > >> scheduler
> > > > >>
> > > > >> > > > > > algorithm [in management node].
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > Thanks in advance
> > > > >> > > > > >
> > > > >> > > > > > Best regards,
> > > > >> > > > > > Phd team
> > > > >> > > > >
> > > > >> > > > > - --
> > > > >> > > > > - -------------------------------
> > > > >> > > > > Josh Thompson
> > > > >> > > > > VCL Developer
> > > > >> > > > > North Carolina State University
> > > > >> > > > >
> > > > >> > > > > my GPG/PGP key can be found at pgp.mit.edu
> > > > >> > > > >
> > > > >> > > > > All electronic mail messages in connection with State
> > > > >> > > > > business
> > > > >>
> > > > >> which
> > > > >>
> > > > >> > > > > are sent to or received by this account are subject to
> the NC
> > > > >>
> > > > >> Public
> > > > >>
> > > > >> > > > > Records Law and may be disclosed to third parties.
> > > > >> > > > > -----BEGIN PGP SIGNATURE-----
> > > > >> > > > > Version: GnuPG v2.0.22 (GNU/Linux)
> > >
> > > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
> > >
> > > > >> > > > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
> > > > >> > > > > =RMTU
> > > > >> > > > > -----END PGP SIGNATURE-----
> > > > >> > >
> > > > >> > > - --
> > > > >> > > - -------------------------------
> > > > >> > > Josh Thompson
> > > > >> > > VCL Developer
> > > > >> > > North Carolina State University
> > > > >> > >
> > > > >> > > my GPG/PGP key can be found at pgp.mit.edu
> > > > >> > >
> > > > >> > > All electronic mail messages in connection with State business
> > >
> > > which
> > >
> > > > >> > > are sent to or received by this account are subject to the NC
> > >
> > > Public
> > >
> > > > >> > > Records Law and may be disclosed to third parties.
> > > > >> > > -----BEGIN PGP SIGNATURE-----
> > > > >> > > Version: GnuPG v2.0.22 (GNU/Linux)
> > > > >> > >
> > > > >> > >
> iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx
> > > > >> > > 5uQAniwpvKVjVGdW//rvNBYCKu485Siy
> > > > >> > > =xWum
> > > > >> > > -----END PGP SIGNATURE-----
> > > > >>
> > > > >> - --
> > > > >> - -------------------------------
> > > > >> Josh Thompson
> > > > >> VCL Developer
> > > > >> North Carolina State University
> > > > >>
> > > > >> my GPG/PGP key can be found at pgp.mit.edu
> > > > >>
> > > > >> All electronic mail messages in connection with State business
> which
> > > > >> are sent to or received by this account are subject to the NC
> Public
> > > > >> Records Law and may be disclosed to third parties.
> > > > >> -----BEGIN PGP SIGNATURE-----
> > > > >> Version: GnuPG v2.0.22 (GNU/Linux)
> > > > >>
> > > > >> iEYEARECAAYFAlRSieYACgkQV/LQcNdtPQOjlACfS97i9Hk0Tmspy+6vfgOcENht
> > > > >> UtYAn008A0w5VBq+eg5iGg1GRlff3zDB
> > > > >> =tCAJ
> > > > >> -----END PGP SIGNATURE-----
> > >
> > > - --
> > > - -------------------------------
> > > Josh Thompson
> > > VCL Developer
> > > North Carolina State University
> > >
> > > my GPG/PGP key can be found at pgp.mit.edu
> > >
> > > All electronic mail messages in connection with State business which
> > > are sent to or received by this account are subject to the NC Public
> > > Records Law and may be disclosed to third parties.
> > > -----BEGIN PGP SIGNATURE-----
> > > Version: GnuPG v2.0.22 (GNU/Linux)
> > >
> > > iEYEARECAAYFAlRal+cACgkQV/LQcNdtPQOAGACeL2Sanhs7ShZaCyKcSo5N2BYk
> > > wLEAn08YbcyiCmR4rY9u4kZJocfxY1Ju
> > > =pxdL
> > > -----END PGP SIGNATURE-----
> - --
> - -------------------------------
> Josh Thompson
> VCL Developer
> North Carolina State University
>
> my GPG/PGP key can be found at pgp.mit.edu
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (GNU/Linux)
>
> iEYEARECAAYFAlRdPGkACgkQV/LQcNdtPQNFjACfdZm6YIKYhJ3EtamGXOfX1Dgs
> X90An3OeCGLL58j2VzzA96jTPfBuIDgI
> =w2De
> -----END PGP SIGNATURE-----
>
>
Re: Scheduling in VCL
Posted by Josh Thompson <jo...@ncsu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Khder,
VCL knows about the specs (RAM, cores, processor speed) of each VM host and
the specs of each image. It does not assign a reservation to a VM on a host
that would result in the host's RAM being overbooked. The number of cores are
not currently checked. So, they could be overbooked. Functionality to limit
reservations based on number of cores will be added at a future date. So,
yes, it is the responsibility of the hypervisor to assign the required
resources, but it should never have to overbook the RAM to bring up a VM for a
new reservation. For bare metal reservations, an image is never assigned to a
node that doesn't meet the minimum specs of the image.
Multiple users can make reservations for the same image. A separate node is
deployed for every reservation. There could be a slight impact on deploy time
since VCL uses linked clones for the disk files. So, reads for the same image
would be hitting the same file. Write would be going to separate files. This
is actually somewhat complicated because the base file on which the linked
clones is actually only shared with hosts that are using the same datastore.
When a node is already loaded with an image someone requests (we call this a
hot standby node), the deploy time is obviously much shorter. VCL does not
allow nodes to be shared other than the case of a bare metal machine running a
hypervisor and hosting VMs (the VMs aren't shared, but the physical machine
is). So, only one user would get a given preloaded node.
Josh
On Thursday, November 06, 2014 2:46:59 AM Khder Omar wrote:
> Dear Josh,
>
> Thanks again for your reply ! I understand the mapping part ! Concerning
> the VCL model as you might mentioned the terms could be different comparing
> to a typical IaaS.
> Most probably I was bit confusing the images provisioning. If I understood
> it correctly, once reservation process starts, a new image will be deployed
> in the corresponding computer or host [just streaming the disk images files
> to build a complete environment within specific resources CPU and RAM]
>
> Therefore, that will be the Physical/Virtual deployer [Hypervisor] role to
> assign those resources ?
>
> Another interesting point which highlights the point if many users can make
> more than one reservation for the same image ? does it affect the image
> loading time [queue of requests to access/use shared resource] ? besides,
> if the image is already deployed and running in a specific computer [can be
> physical/virtual], can many users access the same computer at the same time
> ?
>
> Thanks for your clarification !
>
> Best regards,
> Phd Team
>
> 2014-11-05 22:34 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Khder,
> >
> > Mapping is just a way for the scheduler part of the frontend to know what
> > computers a given image can be loaded on.This could be because of an
> > architectural restriction or it could simply be because a researcher
> > bought
> > her own hardware and only her images should be run on them. Mapping
> > doesn't
> > actually have to do with where the image files reside. I don't think the
> > backend code ever even looks at the mapping information.
> >
> > Remember that VCL predates the terms cloud computing, IaaS, PaaS, etc.
> > So,
> > the terms used for VCL might be somewhat different from more recent
> > systems.
> >
> > In VCL, each "image" or "environment" consists of a set of disk image
> > files
> > and the associated information about the image that is in the database.
> > Initial "base images" must be created somewhat manually. Then, further
> > images
> > are derived from those base images. I don't know that VCL really has
> > flavors
> > because each image will have its own CPU and RAM specifications.
> >
> > Does that answer your questions?
> >
> > Josh
> >
> > On Tuesday, November 04, 2014 11:00:01 PM Khder Omar wrote:
> > > Dear Josh,
> > >
> > > I was wondering during the creation of new image, it does not require to
> >
> > be
> >
> > > stored in the computer node itself, it can be stored in an image library
> > > like a SAN or NAS storage then we mapped a given image to a certain
> > > computer or computer group ! the fact to 'map' an image or image group
> >
> > to a
> >
> > > computer or computer group, does it mean 'streaming' the image 'locally'
> >
> > to
> >
> > > the computer node ? or only during the reservation ?
> > > I was bit confused between the terms used in VCL and in a typical IaaS
> > > environment: the image is claimed as a template of a virtual machine, in
> > > VCL, I have noticed that the image is being created from a base OS and
> > > resources information are assigned in the database such as RAM, CPU,
> >
> > arch,
> >
> > > id etc...
> > > Then I might assume [correct me if I understood it wrong] the Virtual
> > > Machine will be initiated [considering an Hypervisor KVM or ESXi ] while
> > > processing the reservation. Therefore, we can describe such image in a
> > > typical IaaS as a virtual machine flavor ready to be deployed and run on
> > > Host or Computer [within an hypervisor supported].
> > >
> > > Thanks!
> > >
> > > PhD team
> > >
> > > 2014-10-30 20:07 GMT+01:00 Khder Omar <kh...@gmail.com>:
> > > > Dear Josh,
> > > >
> > > > That is true ! Thanks again for the clarification ! I will start
> >
> > doing
> >
> > > > some experiments and see what it gives from performance perspective !
> >
> > In
> >
> > > > the other hand, I will keep updating for any proposal of ideas can be
> > > > generated then we can refine the scheduler code and retest again !
> > > >
> > > > Many thanks for sharing such valuable information!
> > > >
> > > > We will keep updated !
> > > >
> > > > Best Regards
> > > > PhD Team
> > > >
> > > > 2014-10-30 19:56 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > > >> -----BEGIN PGP SIGNED MESSAGE-----
> > > >> Hash: SHA1
> > > >>
> > > >> Khder,
> > > >>
> > > >> The scheduler only allocates a computer randomly if
> > > >> SCHEDULER_ALLOCATE_RANDOM_COMPUTER is set to 1 in conf.php, which it
> >
> > is
> >
> > > >> not by
> > > >> default.
> > > >>
> > > >> Yes, you could create pools that each have a weight or ranking
> >
> > assigned.
> >
> > > >> Each
> > > >> computer would be assigned to a pool. Then, when the initial set of
> > > >> computers
> > > >> is generated, they could be ranked by the pool weight/ranks. It
> > > >> would
> > > >> then be
> > > >> up to an administer or external application to manage the weight/rank
> >
> > of
> >
> > > >> each
> > > >> pool.
> > > >>
> > > >> Josh
> > > >>
> > > >> On Thursday, October 30, 2014 7:48:50 PM Khder Omar wrote:
> > > >> > Dear Josh,
> > > >> >
> > > >> > I would like to thank you for the precious hints concerning the
> > > >>
> > > >> scheduling
> > > >>
> > > >> > nature within VCL. I can see that the set of computers is filtered
> >
> > in
> >
> > > >> > advance and then the scheduler will pick up RANDOMLY one of them.
> > > >> > To
> > > >>
> > > >> check
> > > >>
> > > >> > the possibilities to extend the function considering a large scale
> > > >> > environment where we can introduce the 'region' or 'pool' term in
> >
> > the
> >
> > > >> code.
> > > >>
> > > >> > With multiple management nodes, do you think that using the
> > > >> > 'weight'
> > > >> > for
> > > >> > the workload per pool and then per computer therefore will rank it
> >
> > will
> >
> > > >> be
> > > >>
> > > >> > more precise on how to balance the load across the hole
> >
> > infrastructure
> >
> > > >> > ?
> > > >> >
> > > >> > Best Regards,
> > > >> > PhD team
> > > >> >
> > > >> > 2014-10-30 19:28 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > > >> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > >> > > Hash: SHA1
> > > >> > >
> > > >> > > Khder,
> > > >> > >
> > > >> > > Sorry for the late response.
> > > >> > >
> > > >> > > You can certainly experiment with extending the code to include
> >
> > more
> >
> > > >> > > advanced
> > > >> > > scheduling functions. The existing ranking of computers is
> >
> > something
> >
> > > >> like
> > > >>
> > > >> > > this:
> > > >> > >
> > > >> > > 1) generate a set of all computers meeting the minimum
> >
> > requirements
> >
> > > >> of the
> > > >>
> > > >> > > image, that the user has access to, and that the image is mapped
> >
> > to;
> >
> > > >> order
> > > >>
> > > >> > > that list by node specs from low to high
> > > >> > > 2) remove from that set any computers that are already assigned
> > > >> > > reservations
> > > >> > > 3) if the image is virtual, remove any VMs for which the host
> >
> > doesn't
> >
> > > >> have
> > > >>
> > > >> > > enough memory to load the VM (without overbooking)
> > > >> > > 4) assign the new reservation the first computer from that set
> > > >> > > for
> > > >>
> > > >> which
> > > >>
> > > >> > > an
> > > >> > > active management node can be found
> > > >> > >
> > > >> > > There's actually 3 sets of computers being tracked - those
> > > >> > > already
> > > >>
> > > >> loaded
> > > >>
> > > >> > > with
> > > >> > > the image being requested, those set aside for a block allocation
> >
> > of
> >
> > > >> which
> > > >>
> > > >> > > the
> > > >> > > user is a member, and then a set of both of those plus any
> >
> > available
> >
> > > >> > > computers
> > > >> > > that are currently loaded with another image.
> > > >> > >
> > > >> > > There is a configuration option
> >
> > (SCHEDULER_ALLOCATE_RANDOM_COMPUTER)
> >
> > > >> in
> > > >>
> > > >> > > conf.php that allows the sets of computers to be randomized. The
> > > >> > > motivation
> > > >> > > behind this option is for sites that have homogeneous virtual
> > > >>
> > > >> machines and
> > > >>
> > > >> > > hosts. Randomizing the assigned computer should help spread the
> >
> > load
> >
> > > >> > > equally
> > > >> > > among all existing hosts.
> > > >> > >
> > > >> > > I hope that helps.
> > > >> > >
> > > >> > > Josh
> > > >> > >
> > > >> > > On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote:
> > > >> > > > Hi Josh,
> > > >> > > >
> > > >> > > > Thanks for your reaction. I have checked the code source and I
> > > >>
> > > >> think the
> > > >>
> > > >> > > > only function which might fulfill my question is function
> > > >> > >
> > > >> > > allocComputer(...)
> > > >> > >
> > > >> > > > Eventually, the function a piece test schedule code
> > > >> > > >
> > > >> > > > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) {
> > > >> > > >
> > > >> > > > shuffle($blockids);
> > > >> > > > shuffle($currentids);
> > > >> > > > shuffle($computerids);
> > > >> > > >
> > > >> > > > }
> > > >> > > >
> > > >> > > >
> > > >> > > > I might assume if it was correctly understood that VCL
> > > >> > > > scheduler
> > > >>
> > > >> process
> > > >>
> > > >> > > > determines in first place a computer to be assigned to a
> >
> > management
> >
> > > >> node
> > > >>
> > > >> > > > from a given array or table in order otherwise it can be
> >
> > randomly
> >
> > > >> > > > chosen!
> > > >> > > > Then, we can assume that the scheduling decision is still using
> >
> > a
> >
> > > >> basic
> > > >>
> > > >> > > > order as FCFS or a chance/Random order! It that assumed be
> >
> > correct
> >
> > > >> ? if
> > > >>
> > > >> > > so,
> > > >> > >
> > > >> > > > is there a way to extend the code by including more advanced
> > > >>
> > > >> scheduling
> > > >>
> > > >> > > > functions ?
> > > >> > > >
> > > >> > > > Best regards,
> > > >> > > > PhD Team
> > > >> > > >
> > > >> > > > 2014-10-21 16:11 GMT+01:00 Josh Thompson <
> >
> > josh_thompson@ncsu.edu>:
> > > >> > > > > -----BEGIN PGP SIGNED MESSAGE-----
> > > >> > > > > Hash: SHA1
> > > >> > > > >
> > > >> > > > > Khder,
> > > >> > > > >
> > > >> > > > > The scheduling of reservations is actually done in the web
> >
> > code.
> >
> > > >> Have
> > > >>
> > > >> > > a
> > > >> > >
> > > >> > > > > look
> > > >> > > > > at the isAvailable function in the web/.ht-inc/utils.php
> > > >> > > > > file.
> >
> > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
> >
> > > >> > > > > Josh
> > > >> > > > >
> > > >> > > > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
> > > >> > > > > > Dear all,
> > > >> > > > > >
> > > >> > > > > > We were wondering what kind of scheduler algorithms VCL
> >
> > might
> >
> > > >> use ?
> > > >>
> > > >> > > Any
> > > >> > >
> > > >> > > > > > hints about the scheduler source code will be appreciated.
> >
> > The
> >
> > > >> idea
> > > >>
> > > >> > > is
> > > >> > >
> > > >> > > > > > actually to check how VCL will perform while changing the
> > > >>
> > > >> scheduler
> > > >>
> > > >> > > > > > algorithm [in management node].
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > Thanks in advance
> > > >> > > > > >
> > > >> > > > > > Best regards,
> > > >> > > > > > Phd team
> > > >> > > > >
> > > >> > > > > - --
> > > >> > > > > - -------------------------------
> > > >> > > > > Josh Thompson
> > > >> > > > > VCL Developer
> > > >> > > > > North Carolina State University
> > > >> > > > >
> > > >> > > > > my GPG/PGP key can be found at pgp.mit.edu
> > > >> > > > >
> > > >> > > > > All electronic mail messages in connection with State
> > > >> > > > > business
> > > >>
> > > >> which
> > > >>
> > > >> > > > > are sent to or received by this account are subject to the NC
> > > >>
> > > >> Public
> > > >>
> > > >> > > > > Records Law and may be disclosed to third parties.
> > > >> > > > > -----BEGIN PGP SIGNATURE-----
> > > >> > > > > Version: GnuPG v2.0.22 (GNU/Linux)
> >
> > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
> >
> > > >> > > > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
> > > >> > > > > =RMTU
> > > >> > > > > -----END PGP SIGNATURE-----
> > > >> > >
> > > >> > > - --
> > > >> > > - -------------------------------
> > > >> > > Josh Thompson
> > > >> > > VCL Developer
> > > >> > > North Carolina State University
> > > >> > >
> > > >> > > my GPG/PGP key can be found at pgp.mit.edu
> > > >> > >
> > > >> > > All electronic mail messages in connection with State business
> >
> > which
> >
> > > >> > > are sent to or received by this account are subject to the NC
> >
> > Public
> >
> > > >> > > Records Law and may be disclosed to third parties.
> > > >> > > -----BEGIN PGP SIGNATURE-----
> > > >> > > Version: GnuPG v2.0.22 (GNU/Linux)
> > > >> > >
> > > >> > > iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx
> > > >> > > 5uQAniwpvKVjVGdW//rvNBYCKu485Siy
> > > >> > > =xWum
> > > >> > > -----END PGP SIGNATURE-----
> > > >>
> > > >> - --
> > > >> - -------------------------------
> > > >> Josh Thompson
> > > >> VCL Developer
> > > >> North Carolina State University
> > > >>
> > > >> my GPG/PGP key can be found at pgp.mit.edu
> > > >>
> > > >> All electronic mail messages in connection with State business which
> > > >> are sent to or received by this account are subject to the NC Public
> > > >> Records Law and may be disclosed to third parties.
> > > >> -----BEGIN PGP SIGNATURE-----
> > > >> Version: GnuPG v2.0.22 (GNU/Linux)
> > > >>
> > > >> iEYEARECAAYFAlRSieYACgkQV/LQcNdtPQOjlACfS97i9Hk0Tmspy+6vfgOcENht
> > > >> UtYAn008A0w5VBq+eg5iGg1GRlff3zDB
> > > >> =tCAJ
> > > >> -----END PGP SIGNATURE-----
> >
> > - --
> > - -------------------------------
> > Josh Thompson
> > VCL Developer
> > North Carolina State University
> >
> > my GPG/PGP key can be found at pgp.mit.edu
> >
> > All electronic mail messages in connection with State business which
> > are sent to or received by this account are subject to the NC Public
> > Records Law and may be disclosed to third parties.
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v2.0.22 (GNU/Linux)
> >
> > iEYEARECAAYFAlRal+cACgkQV/LQcNdtPQOAGACeL2Sanhs7ShZaCyKcSo5N2BYk
> > wLEAn08YbcyiCmR4rY9u4kZJocfxY1Ju
> > =pxdL
> > -----END PGP SIGNATURE-----
- --
- -------------------------------
Josh Thompson
VCL Developer
North Carolina State University
my GPG/PGP key can be found at pgp.mit.edu
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iEYEARECAAYFAlRdPGkACgkQV/LQcNdtPQNFjACfdZm6YIKYhJ3EtamGXOfX1Dgs
X90An3OeCGLL58j2VzzA96jTPfBuIDgI
=w2De
-----END PGP SIGNATURE-----
Re: Scheduling in VCL
Posted by Khder Omar <kh...@gmail.com>.
Dear Josh,
Thanks again for your reply ! I understand the mapping part ! Concerning
the VCL model as you might mentioned the terms could be different comparing
to a typical IaaS.
Most probably I was bit confusing the images provisioning. If I understood
it correctly, once reservation process starts, a new image will be deployed
in the corresponding computer or host [just streaming the disk images files
to build a complete environment within specific resources CPU and RAM]
Therefore, that will be the Physical/Virtual deployer [Hypervisor] role to
assign those resources ?
Another interesting point which highlights the point if many users can make
more than one reservation for the same image ? does it affect the image
loading time [queue of requests to access/use shared resource] ? besides,
if the image is already deployed and running in a specific computer [can be
physical/virtual], can many users access the same computer at the same time
?
Thanks for your clarification !
Best regards,
Phd Team
2014-11-05 22:34 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Khder,
>
> Mapping is just a way for the scheduler part of the frontend to know what
> computers a given image can be loaded on.This could be because of an
> architectural restriction or it could simply be because a researcher bought
> her own hardware and only her images should be run on them. Mapping
> doesn't
> actually have to do with where the image files reside. I don't think the
> backend code ever even looks at the mapping information.
>
> Remember that VCL predates the terms cloud computing, IaaS, PaaS, etc. So,
> the terms used for VCL might be somewhat different from more recent
> systems.
>
> In VCL, each "image" or "environment" consists of a set of disk image files
> and the associated information about the image that is in the database.
> Initial "base images" must be created somewhat manually. Then, further
> images
> are derived from those base images. I don't know that VCL really has
> flavors
> because each image will have its own CPU and RAM specifications.
>
> Does that answer your questions?
>
> Josh
>
> On Tuesday, November 04, 2014 11:00:01 PM Khder Omar wrote:
> > Dear Josh,
> >
> > I was wondering during the creation of new image, it does not require to
> be
> > stored in the computer node itself, it can be stored in an image library
> > like a SAN or NAS storage then we mapped a given image to a certain
> > computer or computer group ! the fact to 'map' an image or image group
> to a
> > computer or computer group, does it mean 'streaming' the image 'locally'
> to
> > the computer node ? or only during the reservation ?
> > I was bit confused between the terms used in VCL and in a typical IaaS
> > environment: the image is claimed as a template of a virtual machine, in
> > VCL, I have noticed that the image is being created from a base OS and
> > resources information are assigned in the database such as RAM, CPU,
> arch,
> > id etc...
> > Then I might assume [correct me if I understood it wrong] the Virtual
> > Machine will be initiated [considering an Hypervisor KVM or ESXi ] while
> > processing the reservation. Therefore, we can describe such image in a
> > typical IaaS as a virtual machine flavor ready to be deployed and run on
> > Host or Computer [within an hypervisor supported].
> >
> > Thanks!
> >
> > PhD team
> >
> > 2014-10-30 20:07 GMT+01:00 Khder Omar <kh...@gmail.com>:
> > > Dear Josh,
> > >
> > > That is true ! Thanks again for the clarification ! I will start
> doing
> > > some experiments and see what it gives from performance perspective !
> In
> > > the other hand, I will keep updating for any proposal of ideas can be
> > > generated then we can refine the scheduler code and retest again !
> > >
> > > Many thanks for sharing such valuable information!
> > >
> > > We will keep updated !
> > >
> > > Best Regards
> > > PhD Team
> > >
> > > 2014-10-30 19:56 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > >> -----BEGIN PGP SIGNED MESSAGE-----
> > >> Hash: SHA1
> > >>
> > >> Khder,
> > >>
> > >> The scheduler only allocates a computer randomly if
> > >> SCHEDULER_ALLOCATE_RANDOM_COMPUTER is set to 1 in conf.php, which it
> is
> > >> not by
> > >> default.
> > >>
> > >> Yes, you could create pools that each have a weight or ranking
> assigned.
> > >> Each
> > >> computer would be assigned to a pool. Then, when the initial set of
> > >> computers
> > >> is generated, they could be ranked by the pool weight/ranks. It would
> > >> then be
> > >> up to an administer or external application to manage the weight/rank
> of
> > >> each
> > >> pool.
> > >>
> > >> Josh
> > >>
> > >> On Thursday, October 30, 2014 7:48:50 PM Khder Omar wrote:
> > >> > Dear Josh,
> > >> >
> > >> > I would like to thank you for the precious hints concerning the
> > >>
> > >> scheduling
> > >>
> > >> > nature within VCL. I can see that the set of computers is filtered
> in
> > >> > advance and then the scheduler will pick up RANDOMLY one of them. To
> > >>
> > >> check
> > >>
> > >> > the possibilities to extend the function considering a large scale
> > >> > environment where we can introduce the 'region' or 'pool' term in
> the
> > >>
> > >> code.
> > >>
> > >> > With multiple management nodes, do you think that using the 'weight'
> > >> > for
> > >> > the workload per pool and then per computer therefore will rank it
> will
> > >>
> > >> be
> > >>
> > >> > more precise on how to balance the load across the hole
> infrastructure
> > >> > ?
> > >> >
> > >> > Best Regards,
> > >> > PhD team
> > >> >
> > >> > 2014-10-30 19:28 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > >> > > -----BEGIN PGP SIGNED MESSAGE-----
> > >> > > Hash: SHA1
> > >> > >
> > >> > > Khder,
> > >> > >
> > >> > > Sorry for the late response.
> > >> > >
> > >> > > You can certainly experiment with extending the code to include
> more
> > >> > > advanced
> > >> > > scheduling functions. The existing ranking of computers is
> something
> > >>
> > >> like
> > >>
> > >> > > this:
> > >> > >
> > >> > > 1) generate a set of all computers meeting the minimum
> requirements
> > >>
> > >> of the
> > >>
> > >> > > image, that the user has access to, and that the image is mapped
> to;
> > >>
> > >> order
> > >>
> > >> > > that list by node specs from low to high
> > >> > > 2) remove from that set any computers that are already assigned
> > >> > > reservations
> > >> > > 3) if the image is virtual, remove any VMs for which the host
> doesn't
> > >>
> > >> have
> > >>
> > >> > > enough memory to load the VM (without overbooking)
> > >> > > 4) assign the new reservation the first computer from that set for
> > >>
> > >> which
> > >>
> > >> > > an
> > >> > > active management node can be found
> > >> > >
> > >> > > There's actually 3 sets of computers being tracked - those already
> > >>
> > >> loaded
> > >>
> > >> > > with
> > >> > > the image being requested, those set aside for a block allocation
> of
> > >>
> > >> which
> > >>
> > >> > > the
> > >> > > user is a member, and then a set of both of those plus any
> available
> > >> > > computers
> > >> > > that are currently loaded with another image.
> > >> > >
> > >> > > There is a configuration option
> (SCHEDULER_ALLOCATE_RANDOM_COMPUTER)
> > >>
> > >> in
> > >>
> > >> > > conf.php that allows the sets of computers to be randomized. The
> > >> > > motivation
> > >> > > behind this option is for sites that have homogeneous virtual
> > >>
> > >> machines and
> > >>
> > >> > > hosts. Randomizing the assigned computer should help spread the
> load
> > >> > > equally
> > >> > > among all existing hosts.
> > >> > >
> > >> > > I hope that helps.
> > >> > >
> > >> > > Josh
> > >> > >
> > >> > > On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote:
> > >> > > > Hi Josh,
> > >> > > >
> > >> > > > Thanks for your reaction. I have checked the code source and I
> > >>
> > >> think the
> > >>
> > >> > > > only function which might fulfill my question is function
> > >> > >
> > >> > > allocComputer(...)
> > >> > >
> > >> > > > Eventually, the function a piece test schedule code
> > >> > > >
> > >> > > > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) {
> > >> > > >
> > >> > > > shuffle($blockids);
> > >> > > > shuffle($currentids);
> > >> > > > shuffle($computerids);
> > >> > > >
> > >> > > > }
> > >> > > >
> > >> > > >
> > >> > > > I might assume if it was correctly understood that VCL scheduler
> > >>
> > >> process
> > >>
> > >> > > > determines in first place a computer to be assigned to a
> management
> > >>
> > >> node
> > >>
> > >> > > > from a given array or table in order otherwise it can be
> randomly
> > >> > > > chosen!
> > >> > > > Then, we can assume that the scheduling decision is still using
> a
> > >>
> > >> basic
> > >>
> > >> > > > order as FCFS or a chance/Random order! It that assumed be
> correct
> > >>
> > >> ? if
> > >>
> > >> > > so,
> > >> > >
> > >> > > > is there a way to extend the code by including more advanced
> > >>
> > >> scheduling
> > >>
> > >> > > > functions ?
> > >> > > >
> > >> > > > Best regards,
> > >> > > > PhD Team
> > >> > > >
> > >> > > > 2014-10-21 16:11 GMT+01:00 Josh Thompson <
> josh_thompson@ncsu.edu>:
> > >> > > > > -----BEGIN PGP SIGNED MESSAGE-----
> > >> > > > > Hash: SHA1
> > >> > > > >
> > >> > > > > Khder,
> > >> > > > >
> > >> > > > > The scheduling of reservations is actually done in the web
> code.
> > >>
> > >> Have
> > >>
> > >> > > a
> > >> > >
> > >> > > > > look
> > >> > > > > at the isAvailable function in the web/.ht-inc/utils.php file.
> > >> > > > >
> > >> > > > >
> https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
> > >> > > > >
> > >> > > > > Josh
> > >> > > > >
> > >> > > > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
> > >> > > > > > Dear all,
> > >> > > > > >
> > >> > > > > > We were wondering what kind of scheduler algorithms VCL
> might
> > >>
> > >> use ?
> > >>
> > >> > > Any
> > >> > >
> > >> > > > > > hints about the scheduler source code will be appreciated.
> The
> > >>
> > >> idea
> > >>
> > >> > > is
> > >> > >
> > >> > > > > > actually to check how VCL will perform while changing the
> > >>
> > >> scheduler
> > >>
> > >> > > > > > algorithm [in management node].
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > Thanks in advance
> > >> > > > > >
> > >> > > > > > Best regards,
> > >> > > > > > Phd team
> > >> > > > >
> > >> > > > > - --
> > >> > > > > - -------------------------------
> > >> > > > > Josh Thompson
> > >> > > > > VCL Developer
> > >> > > > > North Carolina State University
> > >> > > > >
> > >> > > > > my GPG/PGP key can be found at pgp.mit.edu
> > >> > > > >
> > >> > > > > All electronic mail messages in connection with State business
> > >>
> > >> which
> > >>
> > >> > > > > are sent to or received by this account are subject to the NC
> > >>
> > >> Public
> > >>
> > >> > > > > Records Law and may be disclosed to third parties.
> > >> > > > > -----BEGIN PGP SIGNATURE-----
> > >> > > > > Version: GnuPG v2.0.22 (GNU/Linux)
> > >> > > > >
> > >> > > > >
> iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
> > >> > > > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
> > >> > > > > =RMTU
> > >> > > > > -----END PGP SIGNATURE-----
> > >> > >
> > >> > > - --
> > >> > > - -------------------------------
> > >> > > Josh Thompson
> > >> > > VCL Developer
> > >> > > North Carolina State University
> > >> > >
> > >> > > my GPG/PGP key can be found at pgp.mit.edu
> > >> > >
> > >> > > All electronic mail messages in connection with State business
> which
> > >> > > are sent to or received by this account are subject to the NC
> Public
> > >> > > Records Law and may be disclosed to third parties.
> > >> > > -----BEGIN PGP SIGNATURE-----
> > >> > > Version: GnuPG v2.0.22 (GNU/Linux)
> > >> > >
> > >> > > iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx
> > >> > > 5uQAniwpvKVjVGdW//rvNBYCKu485Siy
> > >> > > =xWum
> > >> > > -----END PGP SIGNATURE-----
> > >>
> > >> - --
> > >> - -------------------------------
> > >> Josh Thompson
> > >> VCL Developer
> > >> North Carolina State University
> > >>
> > >> my GPG/PGP key can be found at pgp.mit.edu
> > >>
> > >> All electronic mail messages in connection with State business which
> > >> are sent to or received by this account are subject to the NC Public
> > >> Records Law and may be disclosed to third parties.
> > >> -----BEGIN PGP SIGNATURE-----
> > >> Version: GnuPG v2.0.22 (GNU/Linux)
> > >>
> > >> iEYEARECAAYFAlRSieYACgkQV/LQcNdtPQOjlACfS97i9Hk0Tmspy+6vfgOcENht
> > >> UtYAn008A0w5VBq+eg5iGg1GRlff3zDB
> > >> =tCAJ
> > >> -----END PGP SIGNATURE-----
> - --
> - -------------------------------
> Josh Thompson
> VCL Developer
> North Carolina State University
>
> my GPG/PGP key can be found at pgp.mit.edu
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (GNU/Linux)
>
> iEYEARECAAYFAlRal+cACgkQV/LQcNdtPQOAGACeL2Sanhs7ShZaCyKcSo5N2BYk
> wLEAn08YbcyiCmR4rY9u4kZJocfxY1Ju
> =pxdL
> -----END PGP SIGNATURE-----
>
>
Re: Scheduling in VCL
Posted by Josh Thompson <jo...@ncsu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Khder,
Mapping is just a way for the scheduler part of the frontend to know what
computers a given image can be loaded on.This could be because of an
architectural restriction or it could simply be because a researcher bought
her own hardware and only her images should be run on them. Mapping doesn't
actually have to do with where the image files reside. I don't think the
backend code ever even looks at the mapping information.
Remember that VCL predates the terms cloud computing, IaaS, PaaS, etc. So,
the terms used for VCL might be somewhat different from more recent systems.
In VCL, each "image" or "environment" consists of a set of disk image files
and the associated information about the image that is in the database.
Initial "base images" must be created somewhat manually. Then, further images
are derived from those base images. I don't know that VCL really has flavors
because each image will have its own CPU and RAM specifications.
Does that answer your questions?
Josh
On Tuesday, November 04, 2014 11:00:01 PM Khder Omar wrote:
> Dear Josh,
>
> I was wondering during the creation of new image, it does not require to be
> stored in the computer node itself, it can be stored in an image library
> like a SAN or NAS storage then we mapped a given image to a certain
> computer or computer group ! the fact to 'map' an image or image group to a
> computer or computer group, does it mean 'streaming' the image 'locally' to
> the computer node ? or only during the reservation ?
> I was bit confused between the terms used in VCL and in a typical IaaS
> environment: the image is claimed as a template of a virtual machine, in
> VCL, I have noticed that the image is being created from a base OS and
> resources information are assigned in the database such as RAM, CPU, arch,
> id etc...
> Then I might assume [correct me if I understood it wrong] the Virtual
> Machine will be initiated [considering an Hypervisor KVM or ESXi ] while
> processing the reservation. Therefore, we can describe such image in a
> typical IaaS as a virtual machine flavor ready to be deployed and run on
> Host or Computer [within an hypervisor supported].
>
> Thanks!
>
> PhD team
>
> 2014-10-30 20:07 GMT+01:00 Khder Omar <kh...@gmail.com>:
> > Dear Josh,
> >
> > That is true ! Thanks again for the clarification ! I will start doing
> > some experiments and see what it gives from performance perspective ! In
> > the other hand, I will keep updating for any proposal of ideas can be
> > generated then we can refine the scheduler code and retest again !
> >
> > Many thanks for sharing such valuable information!
> >
> > We will keep updated !
> >
> > Best Regards
> > PhD Team
> >
> > 2014-10-30 19:56 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA1
> >>
> >> Khder,
> >>
> >> The scheduler only allocates a computer randomly if
> >> SCHEDULER_ALLOCATE_RANDOM_COMPUTER is set to 1 in conf.php, which it is
> >> not by
> >> default.
> >>
> >> Yes, you could create pools that each have a weight or ranking assigned.
> >> Each
> >> computer would be assigned to a pool. Then, when the initial set of
> >> computers
> >> is generated, they could be ranked by the pool weight/ranks. It would
> >> then be
> >> up to an administer or external application to manage the weight/rank of
> >> each
> >> pool.
> >>
> >> Josh
> >>
> >> On Thursday, October 30, 2014 7:48:50 PM Khder Omar wrote:
> >> > Dear Josh,
> >> >
> >> > I would like to thank you for the precious hints concerning the
> >>
> >> scheduling
> >>
> >> > nature within VCL. I can see that the set of computers is filtered in
> >> > advance and then the scheduler will pick up RANDOMLY one of them. To
> >>
> >> check
> >>
> >> > the possibilities to extend the function considering a large scale
> >> > environment where we can introduce the 'region' or 'pool' term in the
> >>
> >> code.
> >>
> >> > With multiple management nodes, do you think that using the 'weight'
> >> > for
> >> > the workload per pool and then per computer therefore will rank it will
> >>
> >> be
> >>
> >> > more precise on how to balance the load across the hole infrastructure
> >> > ?
> >> >
> >> > Best Regards,
> >> > PhD team
> >> >
> >> > 2014-10-30 19:28 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> >> > > -----BEGIN PGP SIGNED MESSAGE-----
> >> > > Hash: SHA1
> >> > >
> >> > > Khder,
> >> > >
> >> > > Sorry for the late response.
> >> > >
> >> > > You can certainly experiment with extending the code to include more
> >> > > advanced
> >> > > scheduling functions. The existing ranking of computers is something
> >>
> >> like
> >>
> >> > > this:
> >> > >
> >> > > 1) generate a set of all computers meeting the minimum requirements
> >>
> >> of the
> >>
> >> > > image, that the user has access to, and that the image is mapped to;
> >>
> >> order
> >>
> >> > > that list by node specs from low to high
> >> > > 2) remove from that set any computers that are already assigned
> >> > > reservations
> >> > > 3) if the image is virtual, remove any VMs for which the host doesn't
> >>
> >> have
> >>
> >> > > enough memory to load the VM (without overbooking)
> >> > > 4) assign the new reservation the first computer from that set for
> >>
> >> which
> >>
> >> > > an
> >> > > active management node can be found
> >> > >
> >> > > There's actually 3 sets of computers being tracked - those already
> >>
> >> loaded
> >>
> >> > > with
> >> > > the image being requested, those set aside for a block allocation of
> >>
> >> which
> >>
> >> > > the
> >> > > user is a member, and then a set of both of those plus any available
> >> > > computers
> >> > > that are currently loaded with another image.
> >> > >
> >> > > There is a configuration option (SCHEDULER_ALLOCATE_RANDOM_COMPUTER)
> >>
> >> in
> >>
> >> > > conf.php that allows the sets of computers to be randomized. The
> >> > > motivation
> >> > > behind this option is for sites that have homogeneous virtual
> >>
> >> machines and
> >>
> >> > > hosts. Randomizing the assigned computer should help spread the load
> >> > > equally
> >> > > among all existing hosts.
> >> > >
> >> > > I hope that helps.
> >> > >
> >> > > Josh
> >> > >
> >> > > On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote:
> >> > > > Hi Josh,
> >> > > >
> >> > > > Thanks for your reaction. I have checked the code source and I
> >>
> >> think the
> >>
> >> > > > only function which might fulfill my question is function
> >> > >
> >> > > allocComputer(...)
> >> > >
> >> > > > Eventually, the function a piece test schedule code
> >> > > >
> >> > > > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) {
> >> > > >
> >> > > > shuffle($blockids);
> >> > > > shuffle($currentids);
> >> > > > shuffle($computerids);
> >> > > >
> >> > > > }
> >> > > >
> >> > > >
> >> > > > I might assume if it was correctly understood that VCL scheduler
> >>
> >> process
> >>
> >> > > > determines in first place a computer to be assigned to a management
> >>
> >> node
> >>
> >> > > > from a given array or table in order otherwise it can be randomly
> >> > > > chosen!
> >> > > > Then, we can assume that the scheduling decision is still using a
> >>
> >> basic
> >>
> >> > > > order as FCFS or a chance/Random order! It that assumed be correct
> >>
> >> ? if
> >>
> >> > > so,
> >> > >
> >> > > > is there a way to extend the code by including more advanced
> >>
> >> scheduling
> >>
> >> > > > functions ?
> >> > > >
> >> > > > Best regards,
> >> > > > PhD Team
> >> > > >
> >> > > > 2014-10-21 16:11 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> >> > > > > -----BEGIN PGP SIGNED MESSAGE-----
> >> > > > > Hash: SHA1
> >> > > > >
> >> > > > > Khder,
> >> > > > >
> >> > > > > The scheduling of reservations is actually done in the web code.
> >>
> >> Have
> >>
> >> > > a
> >> > >
> >> > > > > look
> >> > > > > at the isAvailable function in the web/.ht-inc/utils.php file.
> >> > > > >
> >> > > > > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
> >> > > > >
> >> > > > > Josh
> >> > > > >
> >> > > > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
> >> > > > > > Dear all,
> >> > > > > >
> >> > > > > > We were wondering what kind of scheduler algorithms VCL might
> >>
> >> use ?
> >>
> >> > > Any
> >> > >
> >> > > > > > hints about the scheduler source code will be appreciated. The
> >>
> >> idea
> >>
> >> > > is
> >> > >
> >> > > > > > actually to check how VCL will perform while changing the
> >>
> >> scheduler
> >>
> >> > > > > > algorithm [in management node].
> >> > > > > >
> >> > > > > >
> >> > > > > > Thanks in advance
> >> > > > > >
> >> > > > > > Best regards,
> >> > > > > > Phd team
> >> > > > >
> >> > > > > - --
> >> > > > > - -------------------------------
> >> > > > > Josh Thompson
> >> > > > > VCL Developer
> >> > > > > North Carolina State University
> >> > > > >
> >> > > > > my GPG/PGP key can be found at pgp.mit.edu
> >> > > > >
> >> > > > > All electronic mail messages in connection with State business
> >>
> >> which
> >>
> >> > > > > are sent to or received by this account are subject to the NC
> >>
> >> Public
> >>
> >> > > > > Records Law and may be disclosed to third parties.
> >> > > > > -----BEGIN PGP SIGNATURE-----
> >> > > > > Version: GnuPG v2.0.22 (GNU/Linux)
> >> > > > >
> >> > > > > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
> >> > > > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
> >> > > > > =RMTU
> >> > > > > -----END PGP SIGNATURE-----
> >> > >
> >> > > - --
> >> > > - -------------------------------
> >> > > Josh Thompson
> >> > > VCL Developer
> >> > > North Carolina State University
> >> > >
> >> > > my GPG/PGP key can be found at pgp.mit.edu
> >> > >
> >> > > All electronic mail messages in connection with State business which
> >> > > are sent to or received by this account are subject to the NC Public
> >> > > Records Law and may be disclosed to third parties.
> >> > > -----BEGIN PGP SIGNATURE-----
> >> > > Version: GnuPG v2.0.22 (GNU/Linux)
> >> > >
> >> > > iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx
> >> > > 5uQAniwpvKVjVGdW//rvNBYCKu485Siy
> >> > > =xWum
> >> > > -----END PGP SIGNATURE-----
> >>
> >> - --
> >> - -------------------------------
> >> Josh Thompson
> >> VCL Developer
> >> North Carolina State University
> >>
> >> my GPG/PGP key can be found at pgp.mit.edu
> >>
> >> All electronic mail messages in connection with State business which
> >> are sent to or received by this account are subject to the NC Public
> >> Records Law and may be disclosed to third parties.
> >> -----BEGIN PGP SIGNATURE-----
> >> Version: GnuPG v2.0.22 (GNU/Linux)
> >>
> >> iEYEARECAAYFAlRSieYACgkQV/LQcNdtPQOjlACfS97i9Hk0Tmspy+6vfgOcENht
> >> UtYAn008A0w5VBq+eg5iGg1GRlff3zDB
> >> =tCAJ
> >> -----END PGP SIGNATURE-----
- --
- -------------------------------
Josh Thompson
VCL Developer
North Carolina State University
my GPG/PGP key can be found at pgp.mit.edu
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iEYEARECAAYFAlRal+cACgkQV/LQcNdtPQOAGACeL2Sanhs7ShZaCyKcSo5N2BYk
wLEAn08YbcyiCmR4rY9u4kZJocfxY1Ju
=pxdL
-----END PGP SIGNATURE-----
Re: Scheduling in VCL
Posted by Khder Omar <kh...@gmail.com>.
Dear Josh,
I was wondering during the creation of new image, it does not require to be
stored in the computer node itself, it can be stored in an image library
like a SAN or NAS storage then we mapped a given image to a certain
computer or computer group ! the fact to 'map' an image or image group to a
computer or computer group, does it mean 'streaming' the image 'locally' to
the computer node ? or only during the reservation ?
I was bit confused between the terms used in VCL and in a typical IaaS
environment: the image is claimed as a template of a virtual machine, in
VCL, I have noticed that the image is being created from a base OS and
resources information are assigned in the database such as RAM, CPU, arch,
id etc...
Then I might assume [correct me if I understood it wrong] the Virtual
Machine will be initiated [considering an Hypervisor KVM or ESXi ] while
processing the reservation. Therefore, we can describe such image in a
typical IaaS as a virtual machine flavor ready to be deployed and run on
Host or Computer [within an hypervisor supported].
Thanks!
PhD team
2014-10-30 20:07 GMT+01:00 Khder Omar <kh...@gmail.com>:
> Dear Josh,
>
> That is true ! Thanks again for the clarification ! I will start doing
> some experiments and see what it gives from performance perspective ! In
> the other hand, I will keep updating for any proposal of ideas can be
> generated then we can refine the scheduler code and retest again !
>
> Many thanks for sharing such valuable information!
>
> We will keep updated !
>
> Best Regards
> PhD Team
>
> 2014-10-30 19:56 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Khder,
>>
>> The scheduler only allocates a computer randomly if
>> SCHEDULER_ALLOCATE_RANDOM_COMPUTER is set to 1 in conf.php, which it is
>> not by
>> default.
>>
>> Yes, you could create pools that each have a weight or ranking assigned.
>> Each
>> computer would be assigned to a pool. Then, when the initial set of
>> computers
>> is generated, they could be ranked by the pool weight/ranks. It would
>> then be
>> up to an administer or external application to manage the weight/rank of
>> each
>> pool.
>>
>> Josh
>>
>> On Thursday, October 30, 2014 7:48:50 PM Khder Omar wrote:
>> > Dear Josh,
>> >
>> > I would like to thank you for the precious hints concerning the
>> scheduling
>> > nature within VCL. I can see that the set of computers is filtered in
>> > advance and then the scheduler will pick up RANDOMLY one of them. To
>> check
>> > the possibilities to extend the function considering a large scale
>> > environment where we can introduce the 'region' or 'pool' term in the
>> code.
>> > With multiple management nodes, do you think that using the 'weight' for
>> > the workload per pool and then per computer therefore will rank it will
>> be
>> > more precise on how to balance the load across the hole infrastructure ?
>> >
>> > Best Regards,
>> > PhD team
>> >
>> > 2014-10-30 19:28 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
>> > > -----BEGIN PGP SIGNED MESSAGE-----
>> > > Hash: SHA1
>> > >
>> > > Khder,
>> > >
>> > > Sorry for the late response.
>> > >
>> > > You can certainly experiment with extending the code to include more
>> > > advanced
>> > > scheduling functions. The existing ranking of computers is something
>> like
>> > > this:
>> > >
>> > > 1) generate a set of all computers meeting the minimum requirements
>> of the
>> > > image, that the user has access to, and that the image is mapped to;
>> order
>> > > that list by node specs from low to high
>> > > 2) remove from that set any computers that are already assigned
>> > > reservations
>> > > 3) if the image is virtual, remove any VMs for which the host doesn't
>> have
>> > > enough memory to load the VM (without overbooking)
>> > > 4) assign the new reservation the first computer from that set for
>> which
>> > > an
>> > > active management node can be found
>> > >
>> > > There's actually 3 sets of computers being tracked - those already
>> loaded
>> > > with
>> > > the image being requested, those set aside for a block allocation of
>> which
>> > > the
>> > > user is a member, and then a set of both of those plus any available
>> > > computers
>> > > that are currently loaded with another image.
>> > >
>> > > There is a configuration option (SCHEDULER_ALLOCATE_RANDOM_COMPUTER)
>> in
>> > > conf.php that allows the sets of computers to be randomized. The
>> > > motivation
>> > > behind this option is for sites that have homogeneous virtual
>> machines and
>> > > hosts. Randomizing the assigned computer should help spread the load
>> > > equally
>> > > among all existing hosts.
>> > >
>> > > I hope that helps.
>> > >
>> > > Josh
>> > >
>> > > On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote:
>> > > > Hi Josh,
>> > > >
>> > > > Thanks for your reaction. I have checked the code source and I
>> think the
>> > > > only function which might fulfill my question is function
>> > >
>> > > allocComputer(...)
>> > >
>> > > > Eventually, the function a piece test schedule code
>> > > >
>> > > > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) {
>> > > >
>> > > > shuffle($blockids);
>> > > > shuffle($currentids);
>> > > > shuffle($computerids);
>> > > >
>> > > > }
>> > > >
>> > > >
>> > > > I might assume if it was correctly understood that VCL scheduler
>> process
>> > > > determines in first place a computer to be assigned to a management
>> node
>> > > > from a given array or table in order otherwise it can be randomly
>> > > > chosen!
>> > > > Then, we can assume that the scheduling decision is still using a
>> basic
>> > > > order as FCFS or a chance/Random order! It that assumed be correct
>> ? if
>> > >
>> > > so,
>> > >
>> > > > is there a way to extend the code by including more advanced
>> scheduling
>> > > > functions ?
>> > > >
>> > > > Best regards,
>> > > > PhD Team
>> > > >
>> > > > 2014-10-21 16:11 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
>> > > > > -----BEGIN PGP SIGNED MESSAGE-----
>> > > > > Hash: SHA1
>> > > > >
>> > > > > Khder,
>> > > > >
>> > > > > The scheduling of reservations is actually done in the web code.
>> Have
>> > >
>> > > a
>> > >
>> > > > > look
>> > > > > at the isAvailable function in the web/.ht-inc/utils.php file.
>> > > > >
>> > > > > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
>> > > > >
>> > > > > Josh
>> > > > >
>> > > > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
>> > > > > > Dear all,
>> > > > > >
>> > > > > > We were wondering what kind of scheduler algorithms VCL might
>> use ?
>> > >
>> > > Any
>> > >
>> > > > > > hints about the scheduler source code will be appreciated. The
>> idea
>> > >
>> > > is
>> > >
>> > > > > > actually to check how VCL will perform while changing the
>> scheduler
>> > > > > > algorithm [in management node].
>> > > > > >
>> > > > > >
>> > > > > > Thanks in advance
>> > > > > >
>> > > > > > Best regards,
>> > > > > > Phd team
>> > > > >
>> > > > > - --
>> > > > > - -------------------------------
>> > > > > Josh Thompson
>> > > > > VCL Developer
>> > > > > North Carolina State University
>> > > > >
>> > > > > my GPG/PGP key can be found at pgp.mit.edu
>> > > > >
>> > > > > All electronic mail messages in connection with State business
>> which
>> > > > > are sent to or received by this account are subject to the NC
>> Public
>> > > > > Records Law and may be disclosed to third parties.
>> > > > > -----BEGIN PGP SIGNATURE-----
>> > > > > Version: GnuPG v2.0.22 (GNU/Linux)
>> > > > >
>> > > > > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
>> > > > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
>> > > > > =RMTU
>> > > > > -----END PGP SIGNATURE-----
>> > >
>> > > - --
>> > > - -------------------------------
>> > > Josh Thompson
>> > > VCL Developer
>> > > North Carolina State University
>> > >
>> > > my GPG/PGP key can be found at pgp.mit.edu
>> > >
>> > > All electronic mail messages in connection with State business which
>> > > are sent to or received by this account are subject to the NC Public
>> > > Records Law and may be disclosed to third parties.
>> > > -----BEGIN PGP SIGNATURE-----
>> > > Version: GnuPG v2.0.22 (GNU/Linux)
>> > >
>> > > iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx
>> > > 5uQAniwpvKVjVGdW//rvNBYCKu485Siy
>> > > =xWum
>> > > -----END PGP SIGNATURE-----
>> - --
>> - -------------------------------
>> Josh Thompson
>> VCL Developer
>> North Carolina State University
>>
>> my GPG/PGP key can be found at pgp.mit.edu
>>
>> All electronic mail messages in connection with State business which
>> are sent to or received by this account are subject to the NC Public
>> Records Law and may be disclosed to third parties.
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v2.0.22 (GNU/Linux)
>>
>> iEYEARECAAYFAlRSieYACgkQV/LQcNdtPQOjlACfS97i9Hk0Tmspy+6vfgOcENht
>> UtYAn008A0w5VBq+eg5iGg1GRlff3zDB
>> =tCAJ
>> -----END PGP SIGNATURE-----
>>
>>
>
Re: Scheduling in VCL
Posted by Khder Omar <kh...@gmail.com>.
Dear Josh,
That is true ! Thanks again for the clarification ! I will start doing
some experiments and see what it gives from performance perspective ! In
the other hand, I will keep updating for any proposal of ideas can be
generated then we can refine the scheduler code and retest again !
Many thanks for sharing such valuable information!
We will keep updated !
Best Regards
PhD Team
2014-10-30 19:56 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Khder,
>
> The scheduler only allocates a computer randomly if
> SCHEDULER_ALLOCATE_RANDOM_COMPUTER is set to 1 in conf.php, which it is
> not by
> default.
>
> Yes, you could create pools that each have a weight or ranking assigned.
> Each
> computer would be assigned to a pool. Then, when the initial set of
> computers
> is generated, they could be ranked by the pool weight/ranks. It would
> then be
> up to an administer or external application to manage the weight/rank of
> each
> pool.
>
> Josh
>
> On Thursday, October 30, 2014 7:48:50 PM Khder Omar wrote:
> > Dear Josh,
> >
> > I would like to thank you for the precious hints concerning the
> scheduling
> > nature within VCL. I can see that the set of computers is filtered in
> > advance and then the scheduler will pick up RANDOMLY one of them. To
> check
> > the possibilities to extend the function considering a large scale
> > environment where we can introduce the 'region' or 'pool' term in the
> code.
> > With multiple management nodes, do you think that using the 'weight' for
> > the workload per pool and then per computer therefore will rank it will
> be
> > more precise on how to balance the load across the hole infrastructure ?
> >
> > Best Regards,
> > PhD team
> >
> > 2014-10-30 19:28 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA1
> > >
> > > Khder,
> > >
> > > Sorry for the late response.
> > >
> > > You can certainly experiment with extending the code to include more
> > > advanced
> > > scheduling functions. The existing ranking of computers is something
> like
> > > this:
> > >
> > > 1) generate a set of all computers meeting the minimum requirements of
> the
> > > image, that the user has access to, and that the image is mapped to;
> order
> > > that list by node specs from low to high
> > > 2) remove from that set any computers that are already assigned
> > > reservations
> > > 3) if the image is virtual, remove any VMs for which the host doesn't
> have
> > > enough memory to load the VM (without overbooking)
> > > 4) assign the new reservation the first computer from that set for
> which
> > > an
> > > active management node can be found
> > >
> > > There's actually 3 sets of computers being tracked - those already
> loaded
> > > with
> > > the image being requested, those set aside for a block allocation of
> which
> > > the
> > > user is a member, and then a set of both of those plus any available
> > > computers
> > > that are currently loaded with another image.
> > >
> > > There is a configuration option (SCHEDULER_ALLOCATE_RANDOM_COMPUTER) in
> > > conf.php that allows the sets of computers to be randomized. The
> > > motivation
> > > behind this option is for sites that have homogeneous virtual machines
> and
> > > hosts. Randomizing the assigned computer should help spread the load
> > > equally
> > > among all existing hosts.
> > >
> > > I hope that helps.
> > >
> > > Josh
> > >
> > > On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote:
> > > > Hi Josh,
> > > >
> > > > Thanks for your reaction. I have checked the code source and I think
> the
> > > > only function which might fulfill my question is function
> > >
> > > allocComputer(...)
> > >
> > > > Eventually, the function a piece test schedule code
> > > >
> > > > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) {
> > > >
> > > > shuffle($blockids);
> > > > shuffle($currentids);
> > > > shuffle($computerids);
> > > >
> > > > }
> > > >
> > > >
> > > > I might assume if it was correctly understood that VCL scheduler
> process
> > > > determines in first place a computer to be assigned to a management
> node
> > > > from a given array or table in order otherwise it can be randomly
> > > > chosen!
> > > > Then, we can assume that the scheduling decision is still using a
> basic
> > > > order as FCFS or a chance/Random order! It that assumed be correct ?
> if
> > >
> > > so,
> > >
> > > > is there a way to extend the code by including more advanced
> scheduling
> > > > functions ?
> > > >
> > > > Best regards,
> > > > PhD Team
> > > >
> > > > 2014-10-21 16:11 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > > > > -----BEGIN PGP SIGNED MESSAGE-----
> > > > > Hash: SHA1
> > > > >
> > > > > Khder,
> > > > >
> > > > > The scheduling of reservations is actually done in the web code.
> Have
> > >
> > > a
> > >
> > > > > look
> > > > > at the isAvailable function in the web/.ht-inc/utils.php file.
> > > > >
> > > > > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
> > > > >
> > > > > Josh
> > > > >
> > > > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
> > > > > > Dear all,
> > > > > >
> > > > > > We were wondering what kind of scheduler algorithms VCL might
> use ?
> > >
> > > Any
> > >
> > > > > > hints about the scheduler source code will be appreciated. The
> idea
> > >
> > > is
> > >
> > > > > > actually to check how VCL will perform while changing the
> scheduler
> > > > > > algorithm [in management node].
> > > > > >
> > > > > >
> > > > > > Thanks in advance
> > > > > >
> > > > > > Best regards,
> > > > > > Phd team
> > > > >
> > > > > - --
> > > > > - -------------------------------
> > > > > Josh Thompson
> > > > > VCL Developer
> > > > > North Carolina State University
> > > > >
> > > > > my GPG/PGP key can be found at pgp.mit.edu
> > > > >
> > > > > All electronic mail messages in connection with State business
> which
> > > > > are sent to or received by this account are subject to the NC
> Public
> > > > > Records Law and may be disclosed to third parties.
> > > > > -----BEGIN PGP SIGNATURE-----
> > > > > Version: GnuPG v2.0.22 (GNU/Linux)
> > > > >
> > > > > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
> > > > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
> > > > > =RMTU
> > > > > -----END PGP SIGNATURE-----
> > >
> > > - --
> > > - -------------------------------
> > > Josh Thompson
> > > VCL Developer
> > > North Carolina State University
> > >
> > > my GPG/PGP key can be found at pgp.mit.edu
> > >
> > > All electronic mail messages in connection with State business which
> > > are sent to or received by this account are subject to the NC Public
> > > Records Law and may be disclosed to third parties.
> > > -----BEGIN PGP SIGNATURE-----
> > > Version: GnuPG v2.0.22 (GNU/Linux)
> > >
> > > iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx
> > > 5uQAniwpvKVjVGdW//rvNBYCKu485Siy
> > > =xWum
> > > -----END PGP SIGNATURE-----
> - --
> - -------------------------------
> Josh Thompson
> VCL Developer
> North Carolina State University
>
> my GPG/PGP key can be found at pgp.mit.edu
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (GNU/Linux)
>
> iEYEARECAAYFAlRSieYACgkQV/LQcNdtPQOjlACfS97i9Hk0Tmspy+6vfgOcENht
> UtYAn008A0w5VBq+eg5iGg1GRlff3zDB
> =tCAJ
> -----END PGP SIGNATURE-----
>
>
Re: Scheduling in VCL
Posted by Josh Thompson <jo...@ncsu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Khder,
The scheduler only allocates a computer randomly if
SCHEDULER_ALLOCATE_RANDOM_COMPUTER is set to 1 in conf.php, which it is not by
default.
Yes, you could create pools that each have a weight or ranking assigned. Each
computer would be assigned to a pool. Then, when the initial set of computers
is generated, they could be ranked by the pool weight/ranks. It would then be
up to an administer or external application to manage the weight/rank of each
pool.
Josh
On Thursday, October 30, 2014 7:48:50 PM Khder Omar wrote:
> Dear Josh,
>
> I would like to thank you for the precious hints concerning the scheduling
> nature within VCL. I can see that the set of computers is filtered in
> advance and then the scheduler will pick up RANDOMLY one of them. To check
> the possibilities to extend the function considering a large scale
> environment where we can introduce the 'region' or 'pool' term in the code.
> With multiple management nodes, do you think that using the 'weight' for
> the workload per pool and then per computer therefore will rank it will be
> more precise on how to balance the load across the hole infrastructure ?
>
> Best Regards,
> PhD team
>
> 2014-10-30 19:28 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Khder,
> >
> > Sorry for the late response.
> >
> > You can certainly experiment with extending the code to include more
> > advanced
> > scheduling functions. The existing ranking of computers is something like
> > this:
> >
> > 1) generate a set of all computers meeting the minimum requirements of the
> > image, that the user has access to, and that the image is mapped to; order
> > that list by node specs from low to high
> > 2) remove from that set any computers that are already assigned
> > reservations
> > 3) if the image is virtual, remove any VMs for which the host doesn't have
> > enough memory to load the VM (without overbooking)
> > 4) assign the new reservation the first computer from that set for which
> > an
> > active management node can be found
> >
> > There's actually 3 sets of computers being tracked - those already loaded
> > with
> > the image being requested, those set aside for a block allocation of which
> > the
> > user is a member, and then a set of both of those plus any available
> > computers
> > that are currently loaded with another image.
> >
> > There is a configuration option (SCHEDULER_ALLOCATE_RANDOM_COMPUTER) in
> > conf.php that allows the sets of computers to be randomized. The
> > motivation
> > behind this option is for sites that have homogeneous virtual machines and
> > hosts. Randomizing the assigned computer should help spread the load
> > equally
> > among all existing hosts.
> >
> > I hope that helps.
> >
> > Josh
> >
> > On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote:
> > > Hi Josh,
> > >
> > > Thanks for your reaction. I have checked the code source and I think the
> > > only function which might fulfill my question is function
> >
> > allocComputer(...)
> >
> > > Eventually, the function a piece test schedule code
> > >
> > > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) {
> > >
> > > shuffle($blockids);
> > > shuffle($currentids);
> > > shuffle($computerids);
> > >
> > > }
> > >
> > >
> > > I might assume if it was correctly understood that VCL scheduler process
> > > determines in first place a computer to be assigned to a management node
> > > from a given array or table in order otherwise it can be randomly
> > > chosen!
> > > Then, we can assume that the scheduling decision is still using a basic
> > > order as FCFS or a chance/Random order! It that assumed be correct ? if
> >
> > so,
> >
> > > is there a way to extend the code by including more advanced scheduling
> > > functions ?
> > >
> > > Best regards,
> > > PhD Team
> > >
> > > 2014-10-21 16:11 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > > > -----BEGIN PGP SIGNED MESSAGE-----
> > > > Hash: SHA1
> > > >
> > > > Khder,
> > > >
> > > > The scheduling of reservations is actually done in the web code. Have
> >
> > a
> >
> > > > look
> > > > at the isAvailable function in the web/.ht-inc/utils.php file.
> > > >
> > > > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
> > > >
> > > > Josh
> > > >
> > > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
> > > > > Dear all,
> > > > >
> > > > > We were wondering what kind of scheduler algorithms VCL might use ?
> >
> > Any
> >
> > > > > hints about the scheduler source code will be appreciated. The idea
> >
> > is
> >
> > > > > actually to check how VCL will perform while changing the scheduler
> > > > > algorithm [in management node].
> > > > >
> > > > >
> > > > > Thanks in advance
> > > > >
> > > > > Best regards,
> > > > > Phd team
> > > >
> > > > - --
> > > > - -------------------------------
> > > > Josh Thompson
> > > > VCL Developer
> > > > North Carolina State University
> > > >
> > > > my GPG/PGP key can be found at pgp.mit.edu
> > > >
> > > > All electronic mail messages in connection with State business which
> > > > are sent to or received by this account are subject to the NC Public
> > > > Records Law and may be disclosed to third parties.
> > > > -----BEGIN PGP SIGNATURE-----
> > > > Version: GnuPG v2.0.22 (GNU/Linux)
> > > >
> > > > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
> > > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
> > > > =RMTU
> > > > -----END PGP SIGNATURE-----
> >
> > - --
> > - -------------------------------
> > Josh Thompson
> > VCL Developer
> > North Carolina State University
> >
> > my GPG/PGP key can be found at pgp.mit.edu
> >
> > All electronic mail messages in connection with State business which
> > are sent to or received by this account are subject to the NC Public
> > Records Law and may be disclosed to third parties.
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v2.0.22 (GNU/Linux)
> >
> > iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx
> > 5uQAniwpvKVjVGdW//rvNBYCKu485Siy
> > =xWum
> > -----END PGP SIGNATURE-----
- --
- -------------------------------
Josh Thompson
VCL Developer
North Carolina State University
my GPG/PGP key can be found at pgp.mit.edu
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iEYEARECAAYFAlRSieYACgkQV/LQcNdtPQOjlACfS97i9Hk0Tmspy+6vfgOcENht
UtYAn008A0w5VBq+eg5iGg1GRlff3zDB
=tCAJ
-----END PGP SIGNATURE-----
Re: Scheduling in VCL
Posted by Khder Omar <kh...@gmail.com>.
Dear Josh,
I would like to thank you for the precious hints concerning the scheduling
nature within VCL. I can see that the set of computers is filtered in
advance and then the scheduler will pick up RANDOMLY one of them. To check
the possibilities to extend the function considering a large scale
environment where we can introduce the 'region' or 'pool' term in the code.
With multiple management nodes, do you think that using the 'weight' for
the workload per pool and then per computer therefore will rank it will be
more precise on how to balance the load across the hole infrastructure ?
Best Regards,
PhD team
2014-10-30 19:28 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Khder,
>
> Sorry for the late response.
>
> You can certainly experiment with extending the code to include more
> advanced
> scheduling functions. The existing ranking of computers is something like
> this:
>
> 1) generate a set of all computers meeting the minimum requirements of the
> image, that the user has access to, and that the image is mapped to; order
> that list by node specs from low to high
> 2) remove from that set any computers that are already assigned
> reservations
> 3) if the image is virtual, remove any VMs for which the host doesn't have
> enough memory to load the VM (without overbooking)
> 4) assign the new reservation the first computer from that set for which an
> active management node can be found
>
> There's actually 3 sets of computers being tracked - those already loaded
> with
> the image being requested, those set aside for a block allocation of which
> the
> user is a member, and then a set of both of those plus any available
> computers
> that are currently loaded with another image.
>
> There is a configuration option (SCHEDULER_ALLOCATE_RANDOM_COMPUTER) in
> conf.php that allows the sets of computers to be randomized. The
> motivation
> behind this option is for sites that have homogeneous virtual machines and
> hosts. Randomizing the assigned computer should help spread the load
> equally
> among all existing hosts.
>
> I hope that helps.
>
> Josh
>
> On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote:
> > Hi Josh,
> >
> > Thanks for your reaction. I have checked the code source and I think the
> > only function which might fulfill my question is function
> allocComputer(...)
> >
> > Eventually, the function a piece test schedule code
> >
> > if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) {
> >
> > shuffle($blockids);
> > shuffle($currentids);
> > shuffle($computerids);
> >
> > }
> >
> >
> > I might assume if it was correctly understood that VCL scheduler process
> > determines in first place a computer to be assigned to a management node
> > from a given array or table in order otherwise it can be randomly chosen!
> > Then, we can assume that the scheduling decision is still using a basic
> > order as FCFS or a chance/Random order! It that assumed be correct ? if
> so,
> > is there a way to extend the code by including more advanced scheduling
> > functions ?
> >
> > Best regards,
> > PhD Team
> >
> > 2014-10-21 16:11 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA1
> > >
> > > Khder,
> > >
> > > The scheduling of reservations is actually done in the web code. Have
> a
> > > look
> > > at the isAvailable function in the web/.ht-inc/utils.php file.
> > >
> > > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
> > >
> > > Josh
> > >
> > > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
> > > > Dear all,
> > > >
> > > > We were wondering what kind of scheduler algorithms VCL might use ?
> Any
> > > > hints about the scheduler source code will be appreciated. The idea
> is
> > > > actually to check how VCL will perform while changing the scheduler
> > > > algorithm [in management node].
> > > >
> > > >
> > > > Thanks in advance
> > > >
> > > > Best regards,
> > > > Phd team
> > >
> > > - --
> > > - -------------------------------
> > > Josh Thompson
> > > VCL Developer
> > > North Carolina State University
> > >
> > > my GPG/PGP key can be found at pgp.mit.edu
> > >
> > > All electronic mail messages in connection with State business which
> > > are sent to or received by this account are subject to the NC Public
> > > Records Law and may be disclosed to third parties.
> > > -----BEGIN PGP SIGNATURE-----
> > > Version: GnuPG v2.0.22 (GNU/Linux)
> > >
> > > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
> > > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
> > > =RMTU
> > > -----END PGP SIGNATURE-----
> - --
> - -------------------------------
> Josh Thompson
> VCL Developer
> North Carolina State University
>
> my GPG/PGP key can be found at pgp.mit.edu
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (GNU/Linux)
>
> iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx
> 5uQAniwpvKVjVGdW//rvNBYCKu485Siy
> =xWum
> -----END PGP SIGNATURE-----
>
>
Re: Scheduling in VCL
Posted by Josh Thompson <jo...@ncsu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Khder,
Sorry for the late response.
You can certainly experiment with extending the code to include more advanced
scheduling functions. The existing ranking of computers is something like
this:
1) generate a set of all computers meeting the minimum requirements of the
image, that the user has access to, and that the image is mapped to; order
that list by node specs from low to high
2) remove from that set any computers that are already assigned reservations
3) if the image is virtual, remove any VMs for which the host doesn't have
enough memory to load the VM (without overbooking)
4) assign the new reservation the first computer from that set for which an
active management node can be found
There's actually 3 sets of computers being tracked - those already loaded with
the image being requested, those set aside for a block allocation of which the
user is a member, and then a set of both of those plus any available computers
that are currently loaded with another image.
There is a configuration option (SCHEDULER_ALLOCATE_RANDOM_COMPUTER) in
conf.php that allows the sets of computers to be randomized. The motivation
behind this option is for sites that have homogeneous virtual machines and
hosts. Randomizing the assigned computer should help spread the load equally
among all existing hosts.
I hope that helps.
Josh
On Tuesday, October 21, 2014 9:01:17 PM Khder Omar wrote:
> Hi Josh,
>
> Thanks for your reaction. I have checked the code source and I think the
> only function which might fulfill my question is function allocComputer(...)
>
> Eventually, the function a piece test schedule code
>
> if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) {
>
> shuffle($blockids);
> shuffle($currentids);
> shuffle($computerids);
>
> }
>
>
> I might assume if it was correctly understood that VCL scheduler process
> determines in first place a computer to be assigned to a management node
> from a given array or table in order otherwise it can be randomly chosen!
> Then, we can assume that the scheduling decision is still using a basic
> order as FCFS or a chance/Random order! It that assumed be correct ? if so,
> is there a way to extend the code by including more advanced scheduling
> functions ?
>
> Best regards,
> PhD Team
>
> 2014-10-21 16:11 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Khder,
> >
> > The scheduling of reservations is actually done in the web code. Have a
> > look
> > at the isAvailable function in the web/.ht-inc/utils.php file.
> >
> > https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
> >
> > Josh
> >
> > On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
> > > Dear all,
> > >
> > > We were wondering what kind of scheduler algorithms VCL might use ? Any
> > > hints about the scheduler source code will be appreciated. The idea is
> > > actually to check how VCL will perform while changing the scheduler
> > > algorithm [in management node].
> > >
> > >
> > > Thanks in advance
> > >
> > > Best regards,
> > > Phd team
> >
> > - --
> > - -------------------------------
> > Josh Thompson
> > VCL Developer
> > North Carolina State University
> >
> > my GPG/PGP key can be found at pgp.mit.edu
> >
> > All electronic mail messages in connection with State business which
> > are sent to or received by this account are subject to the NC Public
> > Records Law and may be disclosed to third parties.
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v2.0.22 (GNU/Linux)
> >
> > iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
> > OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
> > =RMTU
> > -----END PGP SIGNATURE-----
- --
- -------------------------------
Josh Thompson
VCL Developer
North Carolina State University
my GPG/PGP key can be found at pgp.mit.edu
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iEYEARECAAYFAlRSgzoACgkQV/LQcNdtPQMRBgCfaBOvBvUNnfdr5BLR7cnr+vMx
5uQAniwpvKVjVGdW//rvNBYCKu485Siy
=xWum
-----END PGP SIGNATURE-----
Re: Scheduling in VCL
Posted by Khder Omar <kh...@gmail.com>.
Hi Josh,
Thanks for your reaction. I have checked the code source and I think the
only function which might fulfill my question is function allocComputer(...)
Eventually, the function a piece test schedule code
if(SCHEDULER_ALLOCATE_RANDOM_COMPUTER) {
shuffle($blockids);
shuffle($currentids);
shuffle($computerids);
}
I might assume if it was correctly understood that VCL scheduler process
determines in first place a computer to be assigned to a management node
from a given array or table in order otherwise it can be randomly chosen!
Then, we can assume that the scheduling decision is still using a basic
order as FCFS or a chance/Random order! It that assumed be correct ? if so,
is there a way to extend the code by including more advanced scheduling
functions ?
Best regards,
PhD Team
2014-10-21 16:11 GMT+01:00 Josh Thompson <jo...@ncsu.edu>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Khder,
>
> The scheduling of reservations is actually done in the web code. Have a
> look
> at the isAvailable function in the web/.ht-inc/utils.php file.
>
> https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
>
> Josh
>
> On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
> > Dear all,
> >
> > We were wondering what kind of scheduler algorithms VCL might use ? Any
> > hints about the scheduler source code will be appreciated. The idea is
> > actually to check how VCL will perform while changing the scheduler
> > algorithm [in management node].
> >
> >
> > Thanks in advance
> >
> > Best regards,
> > Phd team
> - --
> - -------------------------------
> Josh Thompson
> VCL Developer
> North Carolina State University
>
> my GPG/PGP key can be found at pgp.mit.edu
>
> All electronic mail messages in connection with State business which
> are sent to or received by this account are subject to the NC Public
> Records Law and may be disclosed to third parties.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (GNU/Linux)
>
> iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
> OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
> =RMTU
> -----END PGP SIGNATURE-----
>
>
Re: Scheduling in VCL
Posted by Josh Thompson <jo...@ncsu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Khder,
The scheduling of reservations is actually done in the web code. Have a look
at the isAvailable function in the web/.ht-inc/utils.php file.
https://svn.apache.org/repos/asf/vcl/trunk/web/.ht-inc/utils.php
Josh
On Tuesday, October 21, 2014 4:39:05 PM Khder Omar wrote:
> Dear all,
>
> We were wondering what kind of scheduler algorithms VCL might use ? Any
> hints about the scheduler source code will be appreciated. The idea is
> actually to check how VCL will perform while changing the scheduler
> algorithm [in management node].
>
>
> Thanks in advance
>
> Best regards,
> Phd team
- --
- -------------------------------
Josh Thompson
VCL Developer
North Carolina State University
my GPG/PGP key can be found at pgp.mit.edu
All electronic mail messages in connection with State business which
are sent to or received by this account are subject to the NC Public
Records Law and may be disclosed to third parties.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iEYEARECAAYFAlRGd60ACgkQV/LQcNdtPQPMjQCdF/28fx+VlhmZV0WEMobcv+7p
OAEAn1mct5Iz5bWrLnnX/yQl13wMVwRg
=RMTU
-----END PGP SIGNATURE-----