Python Gump Progress...

[Adam Jack <aj...@trysybase.com>]

Hi,

Sorry I dropped of the face of the planet w/ this. Some M$ virus patches have dorked M$ dial-up and I am w/o a link, and will continue to be (when not in the office) for a while. [Yes, I know, I'm going to get a Linux box, just not this weekend.] Still, w/o interruptions I've made some decent progress w/ Python gump...

Here is some information:

*Internals:

- Created a 'context' tree (GumpContext contains ModuleContext contains ProjectContext) to capture 'work/state' information on top of the model. Supports 'WorkItem' (e.g. a status plus,say. a Cmd/CmdResult pair.)
- Created 'launcher' with Cmd/CmdResult that captures 'exec' information (CWD, ENV, parameters, command, exit_code, output file).
- Create 'logic' with helpers to get projects for an expressions, modules for a list of projects, a build sequence
Note: To cope w/ modules != projects (update modules, build projects) I created code to get a list of modules that contains for a list of projects. 
- Added 'fromaddr' and 'toaddr' to Nag() --- unfortunately the existing 'from' is a keyword clash with Python
- Moved GumpBase to xmlutils.GumpXMLObject [intention to have other non-model XML objects]
- Added more configuration to Workspace (with defaults)

*Processing:

Created integrate.py that does:

 1) Update
 2) Synchronize
 3) Build
 4) Document [to xdocs & launches forrest]
 5) Nag (via e-mail)

... by doing each of these into the context tree, and it growing as the steps occur.

Created display.py to simply print out project metadata in a workspace.

:TODO: Need to process/cope with failures [and stop and/or mark as 'prereq' failed]

*Documentation:

Despite good intentions, I ended up just streaming/writting to files to generate the xdocs, I didn't use GumpXMLObject yet. The code walks down the context tree and generates a hierarchy for the workspace, the modules, and the projects. Right now, the site.xml & book.xml/tab.xml are assumed to pre-exist. 

:TODO: Need help w/ skin/forrest config
:TODO: Need to find proper location for context/xdocs (have it in gump/docs for now)
:TODO: Need to generate 'Statistics' and 'Cross Reference' tabs w/ content.

I am attaching a sample workspace & module & project page, plenty of clean-up to do, but it is a start. Note sure the mailing list will allow HTML to be sent on, but we'll see. No biggee if not.

*Statistics:

Uses simple DBM (dir.work/stats.db) to store success count/failure count/failed prereqs (count)/first build (time)/last build (time) by project name. A simple 'FOG Factor" is generated from these, algorithm could change.

Note: Currently statistics are documented per project, there is not the separate statistics page/tab

:TODO: Need to do cleanup when projects go away

*Testing:

I started some test descriptors and some test programs, but didn't get as far with them as I'd like.

 --------------------------------------------------------------

Overall :TODO: before commit...

:TODO: Need to ensure I didn't break things (RSS/orphans -- not understood.)
:TODO: Doesn't copy cmd output into xdoc. Maybe it could link to it.
:TODO: Doesn't "grep" output like nag did, but uses exit code.
:TODO: Need "-official" to (1) update statistics (2) do nags [maybe -nag]
:TODO: Need "-noforrest" -- forrest runs can take 15mins and up, but I guess GUI is fine for quick builds...
:TODO: Need more testing of existing interfaces (e.g. check.py)
:TODO: Perhaps need non-forrest HTML output (GUI plus forrest plus text might be enough)
:TODO: Need to test as cronjob (ensure logging isn't verbose)
:TODO: Need to cope w/ "gump magic" (some workspace overrides, some nested depends in <ant, that sort of stuff).
:TODO: Need to cope w/ gump packaegs.


Finally:

1) My intentions were only towards completion, I did what I thought best in that vein, I hope that is accepted & these changes are accpeted. I hope I don't put people off w/ the big changes.  This isn't perfectly working/done, but it is one iteration closer.
2) Code review is welcomed (when I submit) I am still learning Python, and know some things aren't pretty or Python-savvy.
3) I made great efforts not to change the existing approaches, and maintained the existing command line interfaces plus GUI interfaces. Also, where at all possible, I re-used existing code and approaches/styles. That said, I've extended pretty much everything -- so please don't get an itch to dive into the Python code before I can submit a mungo patch for review.
4) I hope the forrest approach is accepted, 'cos it is starting to look good. I feel it is worth the requirement to have a forrest install (or somehow be smart and bootstrap off a gump built one) and I hope I can persuade public installers of that. I really don'r feel like manually creating HTML, or even having an XSLT script to pretty up some XML, but I could see how the second could be viable.

regards

Adam