You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@gump.apache.org by Denis Ranger <dr...@optonline.net> on 2003/12/01 18:16:16 UTC
Using Gump to find configurations
I've been dabbling with Gump builds for my own devious purposes for over a year
now. It occurred to me that some of you might be interested in a few
tools/improvements I came up with.
I use Gump for finding all the latest-greatest jars I need for my favorite project of the
day (nowadays using maven). Of course, I want all these jars to be compiled against
each other for the usual reasons. This lead me to problem of figuring out exactly
*what* I needed in term of a profile/workspace definition. After doing it by hand
from scratch once:
- Started from a workspace containing the global 'gump.xml' with all the trimings,
- Added my project (the gump definition kindly generated by maven).
- Ran the generation and looked at the dependency cross-reference. It dawned on me
then that my project should be the only empty entry in the table (no "referenced
by"s). Adding that as a step in the generation should be easy enough to do. And so
I did...
What it boiled down to is this:
- adding a "root" attribute to workspace, containing the name of my project.
- a stylesheet to filter the work/merge.xml by removing any unreferenced project
(except for the root) and any unreferenced module or repository.
- inserting a step in the gen.bat script to apply the stylesheet a certain number of
times (because removing a project may cause other projects to become
unreferenced). Of course, there should be a way to figure out when to stop iterating.
But being lazy, I settled for an iteration variable in the script.
- That worked like a charm and produced:
- an update script that only gets the minimal modules needed.
- a build script that only builds the mimimum projects from those modules.
- a list of the repositories I actually end up using.
I also have a stylesheet that produces the list of packages that should be manually
installed as well as one that produces the global list of jars (generated, included or
installed). I'm currently trying to automate publishing those jars in my local maven
repository.
Sorry for being so long-winded: this is my first post to this list and I had a lot on my
mind... ;-) If anyone is interested in any of the above, I'll be happy to share.
--
Denis Ranger
Gump Statistics (was Re: Using Gump to find configurations)
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
"Denis Ranger" <dr...@optonline.net> wrote:
> - I use the global index and stats only when I'm trying to select which
OS project is
> appropriate (hey, not all xml serializers are created equals...).
Yeah, I'd like Gump to be that also. I started this work 'cos I wanted 'FOG
Factor', for that purpose, and the rest just happened...
> - Stats and Xref... I would prefer an integrated view, like here's all I
know about
> repository X or module Y, or project Z. It would make it more of an
invaluable tool
> when evaluating which java open-source projects to use (already is...)
Do we not have that?
> - I marginally care about *where* a project is stored (repository,
module...) I did at
> some point, though... I seem to recall trying to avoid SourceForge due to
all the
> connection problems I had to their CVS (better now). It would be nice to
have some
> stats about that (connection success/failure rate, lag time)
Yup. I added 'work done on repository' in an attempt to make elapsed time
and success/failure apparent. It was a cheap way to start what you are
sugesting.
I would like to get statistics on performance (and failures) of repositories
some time.
> - I like the FOG factor, although individual numbers and a formula
reminder would
> be useful (for google-impaired users...) Some other measures of stability
would be
> nice too.
I've added this to the per-project FOG Factor page.
> - stats/xrefs by projects are more useful to me than per modules. For
example, while
> it's nice to know that the jakarta-commons module is used so much, it
would be more
> useful to know that it's actually commons-logging that's fashionable (for
example)
Jakarta-commons is an abnormal/extreme case, but I take your point.
Ok, add both per-module and per-project.
These ought show up next time a full run is done.
> - A measure of how far from gumpland a project is would be useful: how
many
> external packages it uses or includes. Is it a new-born project or a
well-established
> one? It tend to favor things I can see and build myself. Well-used
projects are also
> more sellable to clients/collegues.
The Kevin Bacon factor? Degrees of separation? I love the ideas, I wish I
knew how to implement then.
regards,
Adam
Re: Using Gump to find configurations
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
Denis
This is great feedback, thanks. I kinda ran out of steam when it came to
cross referencing, and only skimmed statistics.
Yeah, I was messing around w/ modules not projects to keep the pages small.
Nobody has liked it (not sure even me). I think moving to projects would be
smart.
Basically, to update these things (1) edit the gump/output/statsdb.py and
gump/output/xref.py classes (2) edit the gump/documentation/forrest.py
module (at the bottom).
If you know Java/Perl you can't handle this Python stuff. It is kinda fun,
so long as you don't have to do 8 hour builds to crap out on one line --
that is as painful. Thankfully we now have some simple unit tests (including
for stats and xref).
If you get itch/time to help, please do. Post questions/patches here.
regards
Adam
----- Original Message -----
From: "Denis Ranger" <dr...@optonline.net>
To: "Gump code and data" <gu...@jakarta.apache.org>
Sent: Monday, December 01, 2003 7:29 PM
Subject: Re: Using Gump to find configurations
> Adam,
>
> > I am curious, do you use Gump for cross referincing only, or for Gumping
> > builds? If cross referencing, why would you not use:
>
> Let me clarify. I use gump as a tool for building the jars that I need in
the context of a
> project in particular. For example on one project, I had to used a
third-party library
> which required particular versions of log4j, xerces and xalan. So, add the
things I
> needed to the pot and figure out what else I need (and only that). Once I
have that
> profile, of course, the fun begins and I have to actually build the thing.
This may
> require reverting some modules to prior versions (not necessary the
latest). When I
> get a successful build, I have a *configuration* that ensures I won't be
having any
> weird LinkageErrors due to jar incompatibilities, thus being able to sleep
better at
> night...
>
> What I'm doing now is trying to formalize and instrumentize what I've been
doing in
> an adhoc fashion before (especially since that will be part of my
responsabilities in
> the team I'm about to join).
>
> > http://gump.covalent.com/log/xref.html
>
> Uh... I did the script exactly to avoid manually wading through the
otherwise nice and
> useful global xref...
>
> > Could you give me ideas of what you'd like to see here:
> >
> > http://lsd.student.utwente.nl/gump/gump_xref/index.html
> > http://lsd.student.utwente.nl/gump/gump_xref/repo_module.html
>
> Ok... a few comments, in no particular order:
>
> - I use the global index and stats only when I'm trying to select which
OS project is
> appropriate (hey, not all xml serializers are created equals...).
>
> - Stats and Xref... I would prefer an integrated view, like here's all I
know about
> repository X or module Y, or project Z. It would make it more of an
invaluable tool
> when evaluating which java open-source projects to use (already is...)
>
> - I marginally care about *where* a project is stored (repository,
module...) I did at
> some point, though... I seem to recall trying to avoid SourceForge due to
all the
> connection problems I had to their CVS (better now). It would be nice to
have some
> stats about that (connection success/failure rate, lag time)
>
> - I like the FOG factor, although individual numbers and a formula
reminder would
> be useful (for google-impaired users...) Some other measures of stability
would be
> nice too.
>
> - stats/xrefs by projects are more useful to me than per modules. For
example, while
> it's nice to know that the jakarta-commons module is used so much, it
would be more
> useful to know that it's actually commons-logging that's fashionable (for
example)
>
> - A measure of how far from gumpland a project is would be useful: how
many
> external packages it uses or includes. Is it a new-born project or a
well-established
> one? It tend to favor things I can see and build myself. Well-used
projects are also
> more sellable to clients/collegues.
>
> > If for Gumping builds, isn't there a Maven thing called 'reactor' or
> > something? Does that not work for you?
>
> The reactor is useful if you are building subprojects from a master one.
It also
> assumes that you will be using the latest version, that the subproject is
also
> mavenized. This doesn't work for me because once I get the jars of a
workable
> configuration, I can pretty much forget about the sources (not docs).
>
> > BTW: The Python version of Gump takes a "project expression" (comma
> > separated list of exrepessions with wildcards) to specify what project
to
> > work on. Right now if passed to build.py it builds those projects, and
those
> > below (a full stack). I've been thinking about having Gumpy write a
profile
> > (for a build stack) or have the GUI generate one. Ought be simpel to
add. If
> > you happen to be interested in there, I'd appreciate the help w/
> > development, but this is Python (easy sometimes) not XSLT. Game?
>
> To be honest, I never really looked at the python aspects of gump,
probably because I
> never bothered to learn python... I'm more of a java, perl, php person
nowadays... I'm
> sure I can handle it... (I dropped any mentions to C in my resume over 10
years ago,
> old compiler-guy-is-me) I'll have a look at it, just to humor you... Not
promising
> anything ;)
>
> --
> Denis Ranger
>
Re: Using Gump to find configurations
Posted by Denis Ranger <dr...@optonline.net>.
Adam,
> I am curious, do you use Gump for cross referincing only, or for Gumping
> builds? If cross referencing, why would you not use:
Let me clarify. I use gump as a tool for building the jars that I need in the context of a
project in particular. For example on one project, I had to used a third-party library
which required particular versions of log4j, xerces and xalan. So, add the things I
needed to the pot and figure out what else I need (and only that). Once I have that
profile, of course, the fun begins and I have to actually build the thing. This may
require reverting some modules to prior versions (not necessary the latest). When I
get a successful build, I have a *configuration* that ensures I won't be having any
weird LinkageErrors due to jar incompatibilities, thus being able to sleep better at
night...
What I'm doing now is trying to formalize and instrumentize what I've been doing in
an adhoc fashion before (especially since that will be part of my responsabilities in
the team I'm about to join).
> http://gump.covalent.com/log/xref.html
Uh... I did the script exactly to avoid manually wading through the otherwise nice and
useful global xref...
> Could you give me ideas of what you'd like to see here:
>
> http://lsd.student.utwente.nl/gump/gump_xref/index.html
> http://lsd.student.utwente.nl/gump/gump_xref/repo_module.html
Ok... a few comments, in no particular order:
- I use the global index and stats only when I'm trying to select which OS project is
appropriate (hey, not all xml serializers are created equals...).
- Stats and Xref... I would prefer an integrated view, like here's all I know about
repository X or module Y, or project Z. It would make it more of an invaluable tool
when evaluating which java open-source projects to use (already is...)
- I marginally care about *where* a project is stored (repository, module...) I did at
some point, though... I seem to recall trying to avoid SourceForge due to all the
connection problems I had to their CVS (better now). It would be nice to have some
stats about that (connection success/failure rate, lag time)
- I like the FOG factor, although individual numbers and a formula reminder would
be useful (for google-impaired users...) Some other measures of stability would be
nice too.
- stats/xrefs by projects are more useful to me than per modules. For example, while
it's nice to know that the jakarta-commons module is used so much, it would be more
useful to know that it's actually commons-logging that's fashionable (for example)
- A measure of how far from gumpland a project is would be useful: how many
external packages it uses or includes. Is it a new-born project or a well-established
one? It tend to favor things I can see and build myself. Well-used projects are also
more sellable to clients/collegues.
> If for Gumping builds, isn't there a Maven thing called 'reactor' or
> something? Does that not work for you?
The reactor is useful if you are building subprojects from a master one. It also
assumes that you will be using the latest version, that the subproject is also
mavenized. This doesn't work for me because once I get the jars of a workable
configuration, I can pretty much forget about the sources (not docs).
> BTW: The Python version of Gump takes a "project expression" (comma
> separated list of exrepessions with wildcards) to specify what project to
> work on. Right now if passed to build.py it builds those projects, and those
> below (a full stack). I've been thinking about having Gumpy write a profile
> (for a build stack) or have the GUI generate one. Ought be simpel to add. If
> you happen to be interested in there, I'd appreciate the help w/
> development, but this is Python (easy sometimes) not XSLT. Game?
To be honest, I never really looked at the python aspects of gump, probably because I
never bothered to learn python... I'm more of a java, perl, php person nowadays... I'm
sure I can handle it... (I dropped any mentions to C in my resume over 10 years ago,
old compiler-guy-is-me) I'll have a look at it, just to humor you... Not promising
anything ;)
--
Denis Ranger
Re: Using Gump to find configurations
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
Denis,
I am curious, do you use Gump for cross referincing only, or for Gumping
builds? If cross referencing, why would you not use:
http://gump.covalent.com/log/xref.html
Could you give me ideas of what you'd like to see here:
http://lsd.student.utwente.nl/gump/gump_xref/index.html
http://lsd.student.utwente.nl/gump/gump_xref/repo_module.html
.. I need to focus on cross referencing sometime.
If for Gumping builds, isn't there a Maven thing called 'reactor' or
something? Does that not work for you?
BTW: The Python version of Gump takes a "project expression" (comma
separated list of exrepessions with wildcards) to specify what project to
work on. Right now if passed to build.py it builds those projects, and those
below (a full stack). I've been thinking about having Gumpy write a profile
(for a build stack) or have the GUI generate one. Ought be simpel to add. If
you happen to be interested in there, I'd appreciate the help w/
development, but this is Python (easy sometimes) not XSLT. Game?
regards,
Adam
Re: Using Gump to find configurations
Posted by Nick Chalko <ni...@chalko.com>.
Denis Ranger wrote:
>
> - adding a "root" attribute to workspace, containing the name of my project.
>
>
Very, nice.
I would change root from a attribute to a node called buildable,
which reference all the projects we want to build.
That way I could build the all the projects I have at work using gump.
> - a stylesheet to filter the work/merge.xml by removing any unreferenced project
> (except for the root) and any unreferenced module or repository.
> - inserting a step in the gen.bat script to apply the stylesheet a certain number of
> times (because removing a project may cause other projects to become
> unreferenced). Of course, there should be a way to figure out when to stop iterating.
> But being lazy, I settled for an iteration variable in the script.
> - That worked like a charm and produced:
> - an update script that only gets the minimal modules needed.
> - a build script that only builds the mimimum projects from those modules.
> - a list of the repositories I actually end up using.
>
>I also have a stylesheet that produces the list of packages that should be manually
>installed as well as one that produces the global list of jars (generated, included or
>installed). I'm currently trying to automate publishing those jars in my local maven
>repository.
>
>Sorry for being so long-winded: this is my first post to this list and I had a lot on my
>mind... ;-) If anyone is interested in any of the above, I'll be happy to share.
>--
>Denis Ranger
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: gump-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: gump-help@jakarta.apache.org
>
>