You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@gump.apache.org by "Adam R. B. Jack" <aj...@trysybase.com> on 2004/06/04 06:27:12 UTC
Fw: brutus
Anybody here got sufficient Apache mojo to be trustworthy of root on Brutus?
Sam isn't able to be our sole admin, and I don't think infrastructure want
to take on our issues (if we -- i.e. Stefano/Leo/Stefan?) will take it on.
regards
Adam
----- Original Message -----
From: "Adam R. B. Jack" <aj...@trysybase.com>
To: <in...@apache.org>
Sent: Thursday, June 03, 2004 3:53 PM
Subject: brutus
> Folks,
>
> Some questions regarding Brutus:
>
> 1) Do (sufficient) infrastructure folk have root password on Brutus now?
In
> short, can I post change requests here (not to general@gump & hope Sam is
> looking?)
>
> 2) Do infrastructure folk have any performance metrics/monitoring
software?
> The gump instance on Brutus used to take an hour or so, and now it takes a
> few, however I'm not "in touch w/ the machine's performance" sufficiently
to
> be able to determine if this is a performance problem, or this is just the
> projects that Gump is now running & the work they do.
>
> If there was a slow down (and I can't be sure) it is likely Gump code, but
> events like extra memory, new OS, extra CPU haven't made me 'feel' it is
> faster.
>
> BTW: I run 'top' and see (typically) two things running -- top and
something
> in Gump (python, cvs, java, etc.) I wonder if I ought raise Gump priority
> above other things, or if this is just rude? Ought I by running Gump niced
> lower, since it is so resource intensive?
>
> 3) Is there a way (other than cron) to run something semi-continuously (as
a
> daemon/service), have it restarted if/when it exits, and such? Am I
> describing some init.d related thing? Could anybody provide me with OS
> appropriate pointers? [TIA]
>
> 4) If infrastructure do have root (see 1), would you be willing to (1)
> install tomcat as a service (not user level) (2) alter ProxyPass
> configuration in the Apache HTTPD. I could write up what I'd like to see,
> and submit it via JIRA, for review/comment/queuing.
>
> regards,
>
> Adam
> --
> Experience the Unwired Enterprise:
> http://www.sybase.com/unwiredenterprise
> Try Sybase: http://www.try.sybase.com
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Fw: brutus
Posted by Stefano Mazzocchi <st...@apache.org>.
Stefan Bodewig wrote:
> On Thu, 3 Jun 2004, Adam R. B. Jack <aj...@trysybase.com> wrote:
>
>
>>Anybody here got sufficient Apache mojo to be trustworthy of root on
>>Brutus?
>
>
> Apache mojo, yes. Linux sysadmin experience, yes. Debian experience,
> none. Time, very limited.
>
>
>>Sam isn't able to be our sole admin,
>
>
> And he shouldn't be.
>
> If neither Leo or Stefano want to chime in, I could help. But I'd
> rather be the backup of the backup.
I can't commit enough time to be prompt to act these days, sorry,
deadlines approaching fast.
--
Stefano.
Re: Fw: brutus
Posted by Stefan Bodewig <bo...@apache.org>.
On Thu, 3 Jun 2004, Adam R. B. Jack <aj...@trysybase.com> wrote:
> Anybody here got sufficient Apache mojo to be trustworthy of root on
> Brutus?
Apache mojo, yes. Linux sysadmin experience, yes. Debian experience,
none. Time, very limited.
> Sam isn't able to be our sole admin,
And he shouldn't be.
If neither Leo or Stefano want to chime in, I could help. But I'd
rather be the backup of the backup.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Jira and Gump (Re: Fw: brutus)
Posted by Leo Simons <ls...@jicarilla.org>.
pretty OT, no? :-D. Some yacka-di-yack below.
Jeff Turner wrote:
> On Tue, Jun 08, 2004 at 08:59:17AM -0600, Adam R. B. Jack wrote:
>
>>>I can do it, esp if people can be disciplined about using jira
>>>(too....much....email....).
>>
>>I couldn't agree more. JIRA has to help us smooth out the inflow.
>
> <delurk>
>
> Can you elaborate on this? I'm rewriting JIRA's email templates at the
> moment, so am interested in that general area..
one of the key advantages is that with jira the email it sends you is
always (well, one would hope anyway) laid out the same way. Just like
with a gump nag e-mail, you know the structure is always the same. When
gump sends an "issue fixed" e-mail (was that implemented?) you can mark
the message read based on the subject. You can also filter and search
e-mail that comes from a bot more easily.
With some issues, I can fix them as I see the request come in or
assigned to me, and forget about them (what's missing is of course being
able to mark the changes from my e-mail program; for example a "write
gump success story" button that appears when I'm reading e-mail about
gump; chandler's interesting). With others, I can forget about them
right away and be sure I can still find them when I log into my dashboard.
I think lots and lots of research actually exists about information
overload. Once you build up an e-mail backlog, your inbox keeps just
piling up and up and up. With jira, you know you can just mark a message
read without reading it, since its possible to find that information
later on through the web.
Gump, jira, etc, view 'em as tools for the overworked knowledge worker
to deal with information flow. Anything which allows me to hit 'M' (mark
as read) a split second earlier and forget about a message is progress!
If you were to improve things along these lines....for example, jira
ought to have the resolution in the subject line when an issue is
resolved, since FIXED means I can forget about things immediately whilst
WONTFIX might be a bigger problem.
cheers,
- LSD
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Jira and Gump (Re: Fw: brutus)
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
> > I couldn't agree more. JIRA has to help us smooth out the inflow.
>
> <delurk>
>
> Can you elaborate on this? I'm rewriting JIRA's email templates at the
> moment, so am interested in that general area..
I don't think I was meaning anything more perceptive than JIRA is a good
TODOs list, but I have felt swamped with Gump TODOs recently (we have too
many fun ideas I'd love to find time to do) and I've overstretched my
abilities, and felt overwhelmed. [At least, before I took some days off.
:) ] Basically, JIRA *ought* (if we use discipline) be the repository for
prioritised TODOs, and our 'intray' for tasks. I really ought not do
anything other than what it in JIRA. I need to get that discipline, and I
think the public sharing of tasks/to-dos would be in keeping w/ OSS
principles/benefits.
BTW: I'd kinda like to see 'nag e-mails' from JIRA. Like: "this task has
been waiting a significant time, please either complete it, or assign it to
the pool, or somebody else". Along with that (and for what we'd like here)
we'd like a different group of potential worker per sub-component (in this
case managing the Gump hardware). It'd be nice to have nags to the group
about things sitting in a pool for them.
If we all would read our JIRA dashboard each day we'd not need this, but
that is like expecting folks to read their Gump WWW page each day. Not
feasible. I think in the distributed world of open source nobody has time to
read any WWW site ('cos there are always N). As such , nags e-mails or
RSS/Atom...
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Jira and Gump (Re: Fw: brutus)
Posted by Jeff Turner <je...@apache.org>.
On Tue, Jun 08, 2004 at 08:59:17AM -0600, Adam R. B. Jack wrote:
> > I can do it, esp if people can be disciplined about using jira
> > (too....much....email....).
>
> I couldn't agree more. JIRA has to help us smooth out the inflow.
<delurk>
Can you elaborate on this? I'm rewriting JIRA's email templates at the
moment, so am interested in that general area..
--Jeff
...
> regards,
>
> Adam
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
> For additional commands, e-mail: general-help@gump.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Fw: brutus
Posted by Michael Davey <Mi...@coderage.org>.
Adam R. B. Jack wrote:
>>I can do it, esp if people can be disciplined about using jira
>>(too....much....email....).
>>
>>
>
>I couldn't agree more. JIRA has to help us smooth out the inflow.
>
>
>
>>I think it'll pretty much reduce the time I
>>have available for actual gump development to 0, but that's hardly a
>>change! :/
>>
>>
>
>You know, I find it disappointing that Gump has a high barrier to entry to
>new developers. I know it is there, I feel it, and I know that aspects of
>what I've added to it, contribute to that problem. I'd appreciate all
>pointers/tips on how to make Gump accessible to the newbie.
>
>
Two things I noticed recently:
1. The gump object model is hard to understand at first. Including
examples for each entry would certainly help
2. Perhaps some kind of mechanism to create a hostname.xml file that is
"nearly right" for personal gumps, so that with minimal editing someone
could get a personal gumpp up and running. Contrast, for example, with
Apache webservers httpd.conf. Actually, this probably applies to
project.xml and most of the other gump config files too.
--
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Fw: brutus
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
> I can do it, esp if people can be disciplined about using jira
> (too....much....email....).
I couldn't agree more. JIRA has to help us smooth out the inflow.
> I think it'll pretty much reduce the time I
> have available for actual gump development to 0, but that's hardly a
> change! :/
You know, I find it disappointing that Gump has a high barrier to entry to
new developers. I know it is there, I feel it, and I know that aspects of
what I've added to it, contribute to that problem. I'd appreciate all
pointers/tips on how to make Gump accessible to the newbie.
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Fw: brutus
Posted by Leo Simons <ls...@jicarilla.org>.
Adam R. B. Jack wrote:
> Anybody here got sufficient Apache mojo to be trustworthy of root on Brutus?
> Sam isn't able to be our sole admin, and I don't think infrastructure want
> to take on our issues (if we -- i.e. Stefano/Leo/Stefan?) will take it on.
debian box right?
I can do it, esp if people can be disciplined about using jira
(too....much....email....). I think it'll pretty much reduce the time I
have available for actual gump development to 0, but that's hardly a
change! :/
cheers ;)
- LSD
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Fw: brutus
Posted by Leo Simons <ls...@jicarilla.org>.
Adam R. B. Jack wrote:
> Want me to put things into Gump JIRA?
yes.
> Anybody know if we can have an
> 'Admins' sub-group of Gump in JIRA?
no idea.
- LSD
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Fw: brutus
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
> 3) Figuring out if brutus is busy & performing, or on a go slow. Runs are
> taking 4 or more hours, and I'd love to know why (to try to figure out
what
> to do to try to cut them down.)
Start Date/Time (UTC) Tue, 08 Jun 2004 04:00:41 (UTC)
End Date/Time (UTC) Tue, 08 Jun 2004 11:29:03 (UTC)
A 7 hour build. Now I know there were some network issues (timeouts on mail
to hermes) but I do wish we could have something monitoring Brutus'
performance & reporting to us what occurs (if/when some 'event' at OS level,
occurs.)
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: [RT] Gump service
Posted by Michael Davey <Mi...@coderage.org>.
Adam R. B. Jack wrote:
[snip]
>>Server component to look at the projects and work out which ones Gump
>>needs to try to build next. To start off, the algorithm stays as it is
>>now. In future we can look to detect if there have been CVS commits
>>since last build and so on. Server component puts "n" work pieces into
>>a pool.
>>
>>
[snip]
>>Service/Client component takes one project out of the pool, performs cvs
>>update and attempts to build (again using the current algorithm). Puts
>>the results into a second pool.
>>
>>
>
>I've been thinking about this, but for threads. I recently split Gump into
>pieces with some for of listener & event/request pattern, so we could
>dispatch to unnamed parties. I reworked a 'runner' so we could have various
>types, my goal is one that take 'next available module or project' from a
>list, so we can have multiple concurrent threads working it.
>
>
>
>>View component takes the results from the second pool and produces the
>>web site. Alternatively, the second pool could be replaced with a
>>filestore and the view generates the pages on the fly from the
>>filestore. The view component is forrest with a little glue logic.
>>
>>
>
>I agree, something (lower priority?) could be building a WWW site at the
>same time that real biulds are going on.
>
>I keep thinking we could serialize the context (the build information) to a
>file, but I like having lots of information & haven't found time to
>serialzie it all.
>
>
>
>>The nice thing about this is that it opens up many more possibilities
>>for the future. For instance, it would enable distributed gump at the
>>client component.
>>
>>
>
>Yup, very nice.
>
>My personal goals are for cascading gumps, then multi-threaded (to make use
>of the two CPUs on Brutus), and one day, distributed.
>
>Thanks for the thoughts. Now, how do we make the incremental progress to get
>there?
>
>
I'll respond as no one else has yet. I am very new to Python - I can
play about with simple things like checking of executables but pretty
much anything else is beyond me, so I am unlikely to be able to help
with coding for this. That said...
You said that you have recently split Gump up into pieces using an
event/listener pattern. I infer that the listener works across the
equivalent of my proposed server/client interface but you don't have an
equivalent between client and view. The steps would seem to be:
1a. Find a way of describing a piece of work to the client (a 'type' as
you put it - a project or module). For the moment, I think we can
assume that server and client have a consistent view of the metadata, so
just the module or project name should suffice for the moment.
1b. Rework the runner to work over some form of sockets. The client
will request a piece of work, the server will describe the piece of work
as per [1a] above and mark it as checked out to the client (simply
remove the work piece from its list in the initial implementation).
When the server runs out of work pieces, it starts again from the beginning.
2a. Find a way of describing the results of a build. We need to decide
whether only the log files are to be supplied; log files and location of
the build on the client (assuming it is public access); or log files
plus the actual build.
2b. The client will contact the view and supply the results of the build
as per [2a] above and the view will acknowledge receipt.
2c. Rework the view to understand how to process the results from the
client.
The actual sockets layer could be implemented using ftp, http, svn or
our own protocol.
Later we can add a feedback loop from view to server to let the server
know that the client has supplied the results and make the mechanism
more bulletproof by timing out a client if it doesn't supply the results
in a timely manner (so the work piece can be supplied to a different
client). We could also send the work out to multiple clients to ensure
they agree upon the results.
--
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: [RT] Gump service (was: brutus)
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
Where to start? Ah yes, loved this mail...
> I keep thinking about this (especially as one of my private gumps
> usually takes 25 hours to run).
My first thougt here is cascading Gumps, the main ones store jars in a
public repository, and downstream Gumps (typically private ones) just
use/download the latest Gumped jars. For personal Gump I think this would be
a huge saving.
> The absolute simplest way would be to
> simpy have a bash script that calls gumpy.py in an infinate while loop.
I'd like one that does N 'optiomized' (not yet coded) runs that skipped
things that had not changed, then one official (build it all anyway) run.
Seems easy to code.
> However, I think this would be a very good time to refactor gump into a
> workflow.
Yup.
> Server component to look at the projects and work out which ones Gump
> needs to try to build next. To start off, the algorithm stays as it is
> now. In future we can look to detect if there have been CVS commits
> since last build and so on. Server component puts "n" work pieces into
> a pool.
I thought we had this 'any changes' covered, but found we did not (quite).
When we do the update from CVS|SVN we attempt to detect if changes have
occured.
We mark modules (and underlying projects) with isUpdated() and could simply
build those. [It isn't that simple, but enough for now.]
For CVS the "-q -n update" (and checking for any output) ought do it ('cos
the output ought list changes, since last run). I added that to the CVS
update part. [I don't like scanning for output to detect changes, but I am
no CVS expert, and I do like that this is 'fail safe' (we don't look for
changes since a given time, we look for sinice last update.) I am open to
suggestions on how to do this.
For SVN, this is not an option (unless one parses the output) since it likes
to tell one the current id, even if in sync. I know I tried the 'svn
status --show-updates' but I don't recall if it was enough to tell me
changes, or if I still needed to parse the output.
So ... I wrote a way to check if the 'sync' command found any difference.
Unfortunately I didn't write it unidirectional (it said 'sure there are N
diferences, that build dir you created, etc. etc.) I could attempt to only
report difference from the remote server's copy (adds/dels, not vice verse).
I basically ran out of time on the problem. I would like to revisit it,
because I think there are two good uses for knowing no changes, with your's
here being the most important.
> Service/Client component takes one project out of the pool, performs cvs
> update and attempts to build (again using the current algorithm). Puts
> the results into a second pool.
I've been thinking about this, but for threads. I recently split Gump into
pieces with some for of listener & event/request pattern, so we could
dispatch to unnamed parties. I reworked a 'runner' so we could have various
types, my goal is one that take 'next available module or project' from a
list, so we can have multiple concurrent threads working it.
> View component takes the results from the second pool and produces the
> web site. Alternatively, the second pool could be replaced with a
> filestore and the view generates the pages on the fly from the
> filestore. The view component is forrest with a little glue logic.
I agree, something (lower priority?) could be building a WWW site at the
same time that real biulds are going on.
I keep thinking we could serialize the context (the build information) to a
file, but I like having lots of information & haven't found time to
serialzie it all.
> The nice thing about this is that it opens up many more possibilities
> for the future. For instance, it would enable distributed gump at the
> client component.
Yup, very nice.
My personal goals are for cascading gumps, then multi-threaded (to make use
of the two CPUs on Brutus), and one day, distributed.
Thanks for the thoughts. Now, how do we make the incremental progress to get
there?
regards
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
[RT] Gump service (was: brutus)
Posted by Michael Davey <Mi...@coderage.org>.
Adam R. B. Jack wrote:
>
>4) Figuring out a way (some service?) to have a continuous Gump. The "gump
>every three hours" line in crontab is a waste of two hours when builds take
>4 hours. I'd like some sort of 'permanent restart' loop.
>
>
I keep thinking about this (especially as one of my private gumps
usually takes 25 hours to run). The absolute simplest way would be to
simpy have a bash script that calls gumpy.py in an infinate while loop.
However, I think this would be a very good time to refactor gump into a
workflow.
Server component to look at the projects and work out which ones Gump
needs to try to build next. To start off, the algorithm stays as it is
now. In future we can look to detect if there have been CVS commits
since last build and so on. Server component puts "n" work pieces into
a pool.
Service/Client component takes one project out of the pool, performs cvs
update and attempts to build (again using the current algorithm). Puts
the results into a second pool.
View component takes the results from the second pool and produces the
web site. Alternatively, the second pool could be replaced with a
filestore and the view generates the pages on the fly from the
filestore. The view component is forrest with a little glue logic.
The nice thing about this is that it opens up many more possibilities
for the future. For instance, it would enable distributed gump at the
client component.
--
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Fw: brutus
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
Scott wrote:
> I am a member of the foundation, I helped set up the box in SF, and I
> know apt-get about Debian :)
Cool
> I can do some part-time help. Keeping brutus out of the todo list of
> the infrastructure team is a "good thing".
It seems that Thom May has some plans, but for me I'd appreciate:
1) Fixing this via proxy pass:
http://brutus.apache.org/gump/public/
2) Making tomcat a service (for restarts)
3) Figuring out if brutus is busy & performing, or on a go slow. Runs are
taking 4 or more hours, and I'd love to know why (to try to figure out what
to do to try to cut them down.)
4) Figuring out a way (some service?) to have a continuous Gump. The "gump
every three hours" line in crontab is a waste of two hours when builds take
4 hours. I'd like some sort of 'permanent restart' loop.
Want me to put things into Gump JIRA? Anybody know if we can have an
'Admins' sub-group of Gump in JIRA?
Thanks in advance.
regards
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Fw: brutus
Posted by Scott Sanders <sc...@dotnot.org>.
I am a member of the foundation, I helped set up the box in SF, and I
know apt-get about Debian :)
I can do some part-time help. Keeping brutus out of the todo list of
the infrastructure team is a "good thing".
Scott
Adam R. B. Jack wrote:
> Anybody here got sufficient Apache mojo to be trustworthy of root on Brutus?
> Sam isn't able to be our sole admin, and I don't think infrastructure want
> to take on our issues (if we -- i.e. Stefano/Leo/Stefan?) will take it on.
>
> regards
>
> Adam
> ----- Original Message -----
> From: "Adam R. B. Jack" <aj...@trysybase.com>
> To: <in...@apache.org>
> Sent: Thursday, June 03, 2004 3:53 PM
> Subject: brutus
>
>
>
>>Folks,
>>
>>Some questions regarding Brutus:
>>
>>1) Do (sufficient) infrastructure folk have root password on Brutus now?
>
> In
>
>>short, can I post change requests here (not to general@gump & hope Sam is
>>looking?)
>>
>>2) Do infrastructure folk have any performance metrics/monitoring
>
> software?
>
>>The gump instance on Brutus used to take an hour or so, and now it takes a
>>few, however I'm not "in touch w/ the machine's performance" sufficiently
>
> to
>
>>be able to determine if this is a performance problem, or this is just the
>>projects that Gump is now running & the work they do.
>>
>>If there was a slow down (and I can't be sure) it is likely Gump code, but
>>events like extra memory, new OS, extra CPU haven't made me 'feel' it is
>>faster.
>>
>>BTW: I run 'top' and see (typically) two things running -- top and
>
> something
>
>>in Gump (python, cvs, java, etc.) I wonder if I ought raise Gump priority
>>above other things, or if this is just rude? Ought I by running Gump niced
>>lower, since it is so resource intensive?
>>
>>3) Is there a way (other than cron) to run something semi-continuously (as
>
> a
>
>>daemon/service), have it restarted if/when it exits, and such? Am I
>>describing some init.d related thing? Could anybody provide me with OS
>>appropriate pointers? [TIA]
>>
>>4) If infrastructure do have root (see 1), would you be willing to (1)
>>install tomcat as a service (not user level) (2) alter ProxyPass
>>configuration in the Apache HTTPD. I could write up what I'd like to see,
>>and submit it via JIRA, for review/comment/queuing.
>>
>>regards,
>>
>>Adam
>>--
>>Experience the Unwired Enterprise:
>>http://www.sybase.com/unwiredenterprise
>>Try Sybase: http://www.try.sybase.com
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
> For additional commands, e-mail: general-help@gump.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org