[Fwd: resource usage]

[Leo Simons <ls...@jicarilla.org>]

My address book is a little ****ed up. :(

-------- Original Message --------
Subject: resource usage
Date: Thu, 08 Jul 2004 11:48:43 +0200
From: Leo Simons <ls...@jicarilla.org>
Reply-To: general@incubator.apache.org
To: general@incubator.apache.org
References: <NB...@devtech.com>

Hi gang!

Noel J. Bergman wrote:
> I would also like the GUMP folks to take a close look at their needs.
> Specifically, I have just made arrangements that they ought to be able to
> get a copy of VMware GX for brutus.  That will allow them to clone system
> configurations from a master disk image (virtual disk file) whenever they
> want to start afresh with a clean system.  They could install whichever
> operating system(s) they want in the virtual machines, play around with
> configurations, etc., and not worry about problems.  If a project ended up
> with a requirement came up to do some testing with MS.NET and Mono, they
> could do that, too.  I know that they've looked at resources before, but
> they weren't accounting for disk images and the memory overhead.  They may
> not need to add anything; I'm just suggesting that they check based upon
> this possibility.

VMware would be nice (though I have no experience with GX, I imagine its
better than the consumer stuff ;).

I've been looking into our current resource consumption. Remember, we
used to run gump on a duron ghz PC, and before that it was a pentium II...

1) processor

when there's no gump run, it throts along at 5% or less. During gump
runs it mostly takes about 25% or so (meaning the bottleneck is
elsewhere, probably disk), except during merge steps and things like xml
transformation, when it peaks out briefly (ie a second or two, never
more) at a 100%.

I think we could run two gump builds (like the "main" one and an
"experimental" one) concurrently in seperate VMs and still have quite
acceptable performance, even considering the overhead.

2) memory

We have quite a bit of redundant memory. Gump itself eats about 50mb;
the most intensive java compilation stuff (really big javadoc trees
built using maven in a multiproject setup are an example) never takes
more than a 1/2GB and that's a rare exception I've seen only once, and
not on brutus so far. So the swap space is unused (we have 2GB of memory).

I think if we run two gump builds concurrently in seperate VMs we would
still not need to resort to swap space.

3) disk

Of the 60Gb or so we have we're using about 15GB, and this is with
several seperate gump trees, and including the OS. A single gump tree is
about 4.5GB, with about 125MB of output every night.

Even with the overhead of a GSX install, I guess we have enough space
for a small image (the cvs checkouts 'n stuff don't need to be on the
"master image"), two seperate VMs running at the same time, and /plenty/
of space to spare for gump to grow.

So we're fine wrt space. I think disk speed is our bottleneck right now,
and if we'd consider upgrading anything (which I don't think is
neccessary), it's the main thing to attack. My guess is that the same
would be true for a dedicated nightly build machine.


cheers,


- LSD

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org

RE: resource usage]

["Noel J. Bergman" <no...@devtech.com>]

Adam R. B. Jack wrote:
> We run VMWare here at TrySybase, and have for a couple of years.

> We tried running a full Gump on GSX (on an 'ok' box, not great) and
> we basically brought VMWare down.

> Basiclly thought, Gump pushed GSX too hard for resources.

This might be good to test on brutus, no?

> GSX still resides on top of the operating system (like VMWAre
> Workstation does)

> Memory is the *main* thing that VMWAre go on and on and on about needing
gobs of.

Hence my inquiry to you folks about brutus.  :-)

> We moved from GSX (served us well) to ESX about 6 months ago, and that is
> truly sweet. ESX is based on a Linux Kernal and has no OS sitting between
> it and the box's hardware. Things run very very nicely

Good experience to hear.  If we were to find a similar problem with GSX on
brutus, I expect that VMware would want to know, and we could offer to try
ESX in the same configuration, although I would not take brutus down to do
it.  I'd test it on one of the other x345s first, assuming there is still
one unused.

Anyhow, give it some thought.  :-)

	--- Noel


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org

Re: resource usage]

["Adam R. B. Jack" <aj...@trysybase.com>]

> VMware would be nice (though I have no experience with GX, I imagine its
> better than the consumer stuff ;).

We run VMWare here at TrySybase, and have for a couple of years. All our
demos and such run in VMs (and we have numerous DBs, web servers, app
servers, etc. etc. running on top.) VMWare works nicely for demos where
there isn't continual usage/load.

This posting just brought some things back into memory. We tried running a
full Gump on GSX (on an 'ok' box, not great) and we basically brought VMWare
down. This is probably the only time we've seen VMWare not act like a true
machine (other than this it has been exceptionally stable). [Don't forget,
Gump has had it's sick days & used way too many resources, as we know.] I
can get details of exactly what occured (Robert Meredith, a lurker on infr@)
looked into it. Basiclly thought, Gump pushed GSX too hard for resources.
Virtual Machines are great for flexibility, for sharing, but they can't
compete with raw machines.

Don't forget, GSX still resides on top of the operating system (like VMWAre
Workstation does) so all requests from VMs have to get passed through the
original operating system. Memory is the *main* thing that VMWAre go on and
on and on about needing gobs of. They openly state that VMs are not great
for things like database that use a lot of resources.

We moved from GSX (served us well) to ESX about 6 months ago, and that is
truly sweet. ESX is based on a Linux Kernal and has no OS sitting between it
and the box's hardware. Things run very very nicely, and we are getting
close to 50 VMs on our box (30 before we notice any degradation). Still, it
is a fine balance.

That all said, I'd not discount doing this, just relaying this experience.

regards,

Adam


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org