You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@gump.apache.org by Adam Jack <aj...@TrySybase.com> on 2004/03/25 21:09:33 UTC
Gump Thrashing/Spinning...
I've been trying to run some quick jobs (a check.py not an integrate.py) to
try to get insight into this problem. I'm finding (at least on my machine)
that forrest is growing to huge size, and getting bogged down in swapping.
I can't say this with certainty, but I feel that both Python and Java
degrade very poorly when swapping, not just because they do so much dynamic
memory management. I suspect any background thread for garbage collection is
getting starved. I don't know if this is true, or even if it matters, but we
are getting to a point of resoruce shortage, to the point of outage.
I wonder if Python (Gumpy) and Java (Forrest ) are fighting with each other
for resources. Both are hogs, but I am not sure if being a memory hog is
cause or effect. I don't know if growth is occuring 'cos we finally added
the last straw to the camel's back, or if we have some wierd loop bug. I
suspect forrest is getting stuck on a certain page, but I have no insight
into what is causing this -- nor if the cause is somehow Gumpy. Doing a much
smaller Gump run, hence smaller xdocs set, works w/o problem.
When doing a test run here (without even doign the build parts) I see Gumpy
(the python instance) growing to 136M! I have no clue as to why it would do
that! I wish I could get inside the Python instance to understand where the
memory is, or if it is being leaked. Maybe this is just crowding out
forrest.
Whatever the cause, I am really starting to get 'done' with forrest. I
support it's use, I introduced it & have dealt with the issues and built
workarounds from day one, but it is hard work w/ no fun. I feel it (through
Gumpy trying to dynamically create xdocs) has been the weak/slow link in
Gumpy for a while.
I suspect even the forrest developers here might agree that I took this too
far. Forrest is great for sites, but the gump outputs are getting huge and
not really benefiting from Forrest skins, etc. Maybe I am wrong, maybe I
need to approach the forrest developers (mail to their team) to see what
they think on this.
I wonder if we ought consider replacing Forrest with a pure Python HTML
producer. As above, I can't prove that forrest is the problem, but a pure
Python solution might just halve the unknowns.
Thoughts?
regards,
Adama
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Generating HTML (was Gump Threashing/Spinning)
Posted by Stefano Mazzocchi <st...@apache.org>.
Leo Simons wrote:
>> Now Gump generates it's xdocs using an object tree structure. Watching
>> the
>> python memory grow from 20M (after loading all XML) to 136M (during
>> generating these pages) it has some sort of leak (actual or effective)
>
>
> ouch! Maybe it would pay off to use pipelining (you know, SAX, stuff)
> instead of DOM to generate the object tree.
you are generating a dom on the flight!
ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
please, no matter what you do with forrest, please use a template engine
instead of a DOM. there is no need for a dom for gump.
--
Stefano, thinking seriously that it's time to take a look at the
internals of gumpy
Re: Generating HTML (was Gump Threashing/Spinning)
Posted by Adam Jack <aj...@TrySybase.com>.
> > > Now Gump generates it's xdocs using an object tree structure. Watching
> the
> > > python memory grow from 20M (after loading all XML) to 136M (during
> > > generating these pages) it has some sort of leak (actual or effective)
> >
> > ouch! Maybe it would pay off to use pipelining (you know, SAX, stuff)
> > instead of DOM to generate the object tree.
>
> I wonder if it is some sort of circular dependency (amonst the objects) so
> when I destroy a tree (by pointing the variable to a new one) I wonder if
it
> truely gets destroyed. I know the DOM has an unlink() method for some good
> reason, along these lines.
>
> There is so much thrown up into memory, more with translations to try to
> cope with character sets (and binary junk) and such. I no longer believe
> that any is being thrown away when I mean it to be...
Adding an unlink() to the tree, and calling it, seems to keep memory usage
down to 36M and not 136M. Seems Python needs a hand in recognizing when
things are no longer used.
BTW: It seems I've persuaded forrest to play happily again by removing the
'dependency path' (from cause to project) that I'd added. No clue why, but
whatever.
Gump ran on LSD last night, although it still took a long long time.
However, we are closer to normal again & able to install on Apache hardware
as a valid test.
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Generating HTML (was Gump Threashing/Spinning)
Posted by Adam Jack <aj...@TrySybase.com>.
> > First, I like the dynamic 'tree of nodes' based approach to writing
> > HTML/XML, rather than template
>
> I like merging the concepts. Once you've built the tree, flatten the
> part of it that will make up the page, and feed that to the template
> engine. Even if you don't use a template engine, seperate the flattening
> and splitting from the transformation step.
I am game to explore this. HTML or XML (xdoc) output ought be the same code,
just with different templates, so we can have 'pure Python generating HTML'
and/or 'forrest on xdocs' without us having to compromise, right?
>
> This is different from stuff like anakia, where the template walks the
> tree directly (bad).
I think this is the key. I'd really like to find a good way to go from a
tree of objects (not DOM, our models and context, etc.) to variables for
templates, or whatever. I don't want this to be kludgy, and feel that
cheetah/Python likely have some sort of slick solution.
BTW: I can see the need to use includes, and I can see 'if' directives, but
maybe we'd still want to use bits of templates glued together to get re-use.
I think it depends upon the tree to tempalte choice we make...
> > Second, I didn't realize that Python DOM had nice serialization
mechanism.
> > Maybe I should've used that from the start.
>
> No idea what you're talking about :D
The Python DOM tree will serialize to an XML stream (unlike old DOMs I'm
used to). I wrote some stuff I should never have in Python. Hey, we are here
to learn, right?
> > Now Gump generates it's xdocs using an object tree structure. Watching
the
> > python memory grow from 20M (after loading all XML) to 136M (during
> > generating these pages) it has some sort of leak (actual or effective)
>
> ouch! Maybe it would pay off to use pipelining (you know, SAX, stuff)
> instead of DOM to generate the object tree.
I wonder if it is some sort of circular dependency (amonst the objects) so
when I destroy a tree (by pointing the variable to a new one) I wonder if it
truely gets destroyed. I know the DOM has an unlink() method for some good
reason, along these lines.
There is so much thrown up into memory, more with translations to try to
cope with character sets (and binary junk) and such. I no longer believe
that any is being thrown away when I mean it to be...
> > Fixing this area is clearly important, and I'd appreciate all
insights...
>
> No insights here, just babbling along ;)
Better than my just listening to the babbling in my head. ;-)
regards
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Generating HTML (was Gump Threashing/Spinning)
Posted by Leo Simons <ls...@jicarilla.org>.
Adam Jack wrote:
>>We should probably use a template engine. I'm sure there's a python
>>equivalent for something like velocity (or smarty).
>
> First, I like the dynamic 'tree of nodes' based approach to writing
> HTML/XML, rather than template
I like merging the concepts. Once you've built the tree, flatten the
part of it that will make up the page, and feed that to the template
engine. Even if you don't use a template engine, seperate the flattening
and splitting from the transformation step.
This is different from stuff like anakia, where the template walks the
tree directly (bad).
-- in the main because of pleasant
> experiences with the Perl modules for this in the past. That said, even
> though there are some huge fields here (build output logs)
hehehe. Look here:
http://www.cheetahtemplate.org/docs/users_guide_html_multipage/language.constructs.html
this is what things like #include are for!
> maybe a template
> mechanism is enough for the simple structure (results tables, lists, etc.)
> that we have. We have huge pages, with options missing (or not) based off
> content. I'm not sure any template could cope with that, but I am open
> minded to it.
The biggest ISP in the Netherlands with the biggest content portal in
the Netherlands uses a template engine as a core part of their view
layer (but don't tell 'em ;) :D. At least they used to...
> Second, I didn't realize that Python DOM had nice serialization mechanism.
> Maybe I should've used that from the start.
No idea what you're talking about :D
> Now Gump generates it's xdocs using an object tree structure. Watching the
> python memory grow from 20M (after loading all XML) to 136M (during
> generating these pages) it has some sort of leak (actual or effective)
ouch! Maybe it would pay off to use pipelining (you know, SAX, stuff)
instead of DOM to generate the object tree.
> Fixing this area is clearly important, and I'd appreciate all insights...
No insights here, just babbling along ;)
--
cheers,
- Leo Simons
-----------------------------------------------------------------------
Weblog -- http://leosimons.com/
IoC Component Glue -- http://jicarilla.org/
Articles & Opinions -- http://articles.leosimons.com/
-----------------------------------------------------------------------
"We started off trying to set up a small anarchist community, but
people wouldn't obey the rules."
-- Alan Bennett
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Generating HTML (was Gump Threashing/Spinning)
Posted by Nick Chalko <ni...@chalko.com>.
Perhaps we should run cocoon/fforrest live and just update the xdocs
underneath?
Adam Jack wrote:
>>We should probably use a template engine. I'm sure there's a python
>>equivalent for something like velocity (or smarty).
>>
>>
>
>First, I like the dynamic 'tree of nodes' based approach to writing
>HTML/XML, rather than template -- in the main because of pleasant
>experiences with the Perl modules for this in the past. That said, even
>though there are some huge fields here (build output logs) maybe a template
>mechanism is enough for the simple structure (results tables, lists, etc.)
>that we have. We have huge pages, with options missing (or not) based off
>content. I'm not sure any template could cope with that, but I am open
>minded to it.
>
>Second, I didn't realize that Python DOM had nice serialization mechanism.
>Maybe I should've used that from the start.
>
>Now Gump generates it's xdocs using an object tree structure. Watching the
>python memory grow from 20M (after loading all XML) to 136M (during
>generating these pages) it has some sort of leak (actual or effective):
>
>Fixing this area is clearly important, and I'd appreciate all insights...
>
>regards,
>
>Adam
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
>For additional commands, e-mail: general-help@gump.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Generating HTML (was Gump Threashing/Spinning)
Posted by Adam Jack <aj...@TrySybase.com>.
> We should probably use a template engine. I'm sure there's a python
> equivalent for something like velocity (or smarty).
First, I like the dynamic 'tree of nodes' based approach to writing
HTML/XML, rather than template -- in the main because of pleasant
experiences with the Perl modules for this in the past. That said, even
though there are some huge fields here (build output logs) maybe a template
mechanism is enough for the simple structure (results tables, lists, etc.)
that we have. We have huge pages, with options missing (or not) based off
content. I'm not sure any template could cope with that, but I am open
minded to it.
Second, I didn't realize that Python DOM had nice serialization mechanism.
Maybe I should've used that from the start.
Now Gump generates it's xdocs using an object tree structure. Watching the
python memory grow from 20M (after loading all XML) to 136M (during
generating these pages) it has some sort of leak (actual or effective):
Fixing this area is clearly important, and I'd appreciate all insights...
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump Thrashing/Spinning...
Posted by Sam Ruby <ru...@apache.org>.
Leo Simons wrote:
>
> We should probably use a template engine. I'm sure there's a python
> equivalent for something like velocity (or smarty).
http://www.cheetahtemplate.org/
I use it in my weblogging software.
- Sam Ruby
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump Thrashing/Spinning...
Posted by Leo Simons <ls...@jicarilla.org>.
Adam Jack wrote:
> Whatever the cause, I am really starting to get 'done' with forrest. I
> support it's use, I introduced it & have dealt with the issues and built
> workarounds from day one, but it is hard work w/ no fun.
I remember that feeling. I like the forrest project and I like the
forrest devs a lot, but I've been very frustrated with the forrest
software a few times.
As to why python is a memory hog itself as well -- probably because
variables don't go out of scope and because a lot of things (strings
probably and object representations in the GOM) get copied around.
> I wonder if we ought consider replacing Forrest with a pure Python HTML
> producer. As above, I can't prove that forrest is the problem, but a pure
> Python solution might just halve the unknowns.
>
> Thoughts?
It's a good plan, methinks.
* I think some people would like gump a lot more if you could run it
without needing java at all from a philosophical (free everything) view.
* It's a lot simpler (we really don't need forrest's power).
* It reduces the number of tools one has to be familiar with.
We should probably use a template engine. I'm sure there's a python
equivalent for something like velocity (or smarty).
--
cheers,
- Leo Simons
-----------------------------------------------------------------------
Weblog -- http://leosimons.com/
IoC Component Glue -- http://jicarilla.org/
Articles & Opinions -- http://articles.leosimons.com/
-----------------------------------------------------------------------
"We started off trying to set up a small anarchist community, but
people wouldn't obey the rules."
-- Alan Bennett
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: About ForrestCocoon was: Gump Thrashing/Spinning...
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Adam Jack wrote:
>>Forrest uses cocoon (http://cocoon.apache.org/2.0/)
>>Which is a servlet that does xml pipelines
>>XML -> transform -> transform --> xml (or whatever) out.
>>Along with caching all kinds of other cool stuff.
>>That is what you get when you use the "forrest run" command
>>
>>Because many, many of our pages are never visited this might actually
>>reduce the computational load.
>
> Yup, seems highly likely. It also means that if Forrest barfs on a certain
> page we still get access to the other pages.
Forrest can use simple HTML if you save it as *.ihtml
My suggestion would be to generate a Forrest usable site with ihtml
pages using cheetah, and then publish it so that a dynamic Forrest can
show it nicely skinned.
This way we get the best of all:
1 - use templates
2 - can see pages without running Forrest
3 - don't even need to run static Forrest
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: About ForrestCocoon was: Gump Thrashing/Spinning...
Posted by Adam Jack <aj...@TrySybase.com>.
> Forrest uses cocoon (http://cocoon.apache.org/2.0/)
> Which is a servlet that does xml pipelines
> XML -> transform -> transform --> xml (or whatever) out.
> Along with caching all kinds of other cool stuff.
> That is what you get when you use the "forrest run" command
>
> Because many, many of our pages are never visited this might actually
> reduce the computational load.
Yup, seems highly likely. It also means that if Forrest barfs on a certain
page we still get access to the other pages.
> So for us that means we run tomcat with a forrest install and then just
> push our xdocs to the right location.
> When a users visits a page that the xml underneath has been updated the
> page will regenerate.
Cool. So I assume we simply copy (sync) the xdocs into the 'log' directory,
and not run forrest & copy(sync) outputs. It is a shame that each run will
generate new xdocs, even though the real content may not change, but that
doesn't seem too painful.
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
About ForrestCocoon was: Gump Thrashing/Spinning...
Posted by Nick Chalko <ni...@chalko.com>.
Forrest uses cocoon (http://cocoon.apache.org/2.0/)
Which is a servlet that does xml pipelines
XML -> transform -> transform --> xml (or whatever) out.
Along with caching all kinds of other cool stuff.
That is what you get when you use the "forrest run" command
Because many, many of our pages are never visited this might actually
reduce the computational load.
So for us that means we run tomcat with a forrest install and then just
push our xdocs to the right location.
When a users visits a page that the xml underneath has been updated the
page will regenerate.
R,
Nick
Adam Jack wrote:
>>> 1) use forrest as a dynamic application
>>>
>>>
>
>First, what do you mean by this, please? For those of us who don't know,
>could somebody elaborate?
>
>Second, I think it is forrest that is where we are getting stuck right now.
>Now sure why, but it is locking up. So, if we want forest, we have to figure
>out how to get inside that problem.
>
>regards
>
>Adam
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
>For additional commands, e-mail: general-help@gump.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: [RT] Generator vs Serializer
Posted by Stefano Mazzocchi <st...@apache.org>.
Sam Ruby wrote:
> Adam R. B. Jack wrote:
>
>>> My first proposal was to run forrest dynamic for gump. This means that
>>> you don't have to change anything in the code right now *AND* you get
>>> immediate changes as soon as they are written on disk.
>>
>>
>> I am game to explore alternatives, but this one really attracts me as a
>> short term fix. Can anybody find time to help install this on moof and/or
>> brutus and/or other? I'll give Gump a quick --xdocs option so that it
>> leaves
>> the xdocs it generates are the output [sync'ing/moving to the log
>> directory,
>> correct?], and not try to run Forrest inlined.
>
>
> If somebody can give me a "bill of materials" (preferably in the form of
> a list of URLs of installation instructions), I'll take care of this on
> Brutus.
http://xml.apache.org/forrest/your-project.html#webapp
In short:
1) download forrest
2) unpack
3) type "forrest run"
not that hard. ;-)
--
Stefano.
Re: Installing forrest was: [RT] Generator vs Serializer
Posted by Nick Chalko <ni...@chalko.com>.
Adam R. B. Jack wrote:
>Ok, so help me understand this more. Once we have a webapp installed
>(somehow we have to build it based upon the empty template, right?) we can
>then just squirt new xdocs and/or other content into that directory?
>
>Can we do things like change the site.xml on the fly?
>
Yes
>[i.e. can we copy in
>the new template each time?]
>
yes
> Also, does it have a content with content/xdocs
>below?
>
yes
> Do we simple follow that format and the servlet does the mapping?
>
>
>
Yes
try "forrest run" which starts a jetty on port 8888.
test it by modifiyng the files in build/webapp
>or additional commands, e-mail: general-help@gump.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Installing forrest was: [RT] Generator vs Serializer
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
> http://xml.apache.org/forrest/your-project.html#installing
>
> use the /forrest webapp/ command to create a webapp for deployment and
> then copy that over to the tomcat applications dir.
>
> I think the webapp ends up in build/webapp
Ok, so help me understand this more. Once we have a webapp installed
(somehow we have to build it based upon the empty template, right?) we can
then just squirt new xdocs and/or other content into that directory?
Can we do things like change the site.xml on the fly? [i.e. can we copy in
the new template each time?] Also, does it have a content with content/xdocs
below? Do we simple follow that format and the servlet does the mapping?
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Installing forrest was: [RT] Generator vs Serializer
Posted by Nick Chalko <ni...@chalko.com>.
http://xml.apache.org/forrest/your-project.html#installing
use the /forrest webapp/ command to create a webapp for deployment and
then copy that over to the tomcat applications dir.
I think the webapp ends up in build/webapp
R,
Nick
Sam Ruby wrote:
>
> If somebody can give me a "bill of materials" (preferably in the form
> of a list of URLs of installation instructions), I'll take care of
> this on Brutus.
>
> - Sam Ruby
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
> For additional commands, e-mail: general-help@gump.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: [RT] Generator vs Serializer
Posted by Sam Ruby <ru...@apache.org>.
Adam R. B. Jack wrote:
>>My first proposal was to run forrest dynamic for gump. This means that
>>you don't have to change anything in the code right now *AND* you get
>>immediate changes as soon as they are written on disk.
>
> I am game to explore alternatives, but this one really attracts me as a
> short term fix. Can anybody find time to help install this on moof and/or
> brutus and/or other? I'll give Gump a quick --xdocs option so that it leaves
> the xdocs it generates are the output [sync'ing/moving to the log directory,
> correct?], and not try to run Forrest inlined.
If somebody can give me a "bill of materials" (preferably in the form of
a list of URLs of installation instructions), I'll take care of this on
Brutus.
- Sam Ruby
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: [RT] Generator vs Serializer
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
> My first proposal was to run forrest dynamic for gump. This means that
> you don't have to change anything in the code right now *AND* you get
> immediate changes as soon as they are written on disk.
I am game to explore alternatives, but this one really attracts me as a
short term fix. Can anybody find time to help install this on moof and/or
brutus and/or other? I'll give Gump a quick --xdocs option so that it leaves
the xdocs it generates are the output [sync'ing/moving to the log directory,
correct?], and not try to run Forrest inlined.
If we can have this in place, I think we can look at how we generate
documentation as it becomes available -- and then eventually [unless we have
somebody chomping at the bit for this] allow an HTML (via Cheetah)
alternative to xdocs.
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: [RT] Generator vs Serializer
Posted by Stefano Mazzocchi <st...@apache.org>.
Sam Ruby wrote:
> Adam R. B. Jack wrote:
> >
> > We have gump.document.text, and we could create gump.document.html
> > that use cheetah to write it.
>
> Stefano Mazzocchi wrote:
>
>>
>> +1 in removal of forrest and go plain XHTML + CSS. But please, let's
>> use a velocity-like approach, not a DOM like approach!
>
>
> I may be reading too much into Stefano's words, but if so, the following
> is how I see things.
>
> At the moment, gump.document.* take a complete set of knowledge and
> produce a set of artifacts. The best analogy to Cocoon for this would
> be a serializer which terminates a pipeline.
>
> An alternate approach would be to completely flip this. Have the
> equivalent logic drive the acquisition of certain pieces of information,
> which can be processed as it is being received. The best analogy to
> Cocoon for this would be a generator which is at the beginning of a
> pipeline. Classic gump is closer to this model... after a brief
> generation phase, the execution of the resulting script triggers
> actions, the output of which is intermixed with some static and some
> generated output.
>
> I'm not much into abstract architectural discussion. I tend to focus on
> tangible and measurable quanties that matter to real people. In this
> case, it is clear that Gump is expected to run for a long period of
> time, and I view not having ANY output until EVERYTHING is done as
> something less than ideal.
>
> Producing output as the information becomes available can also
> dramatically reduce working set sizes.
Don't have much time to read email this week so, but I want to kick this
is before things start changing.
Sam is correct about forrest being one massive serializer and gump one
massive generator. The try/fail cycle is way too slow.
This is the usual criticism to forrest.
The usual thing that people miss is that forrest is supposed to be run
"LIVE", that means "dynamic", that means "as a servlet" during
development that is, when try/fail cycle needs to be fast. And it can be
run in batch mode after that (it is able to do deltas and regenerate
only the parts of the site that has changed).
My first proposal was to run forrest dynamic for gump. This means that
you don't have to change anything in the code right now *AND* you get
immediate changes as soon as they are written on disk.
Also, given that we proxypass thru mod_cache it anyway, you don't have
any performance drawback.
As I said, I'm not against XHTML+CSS, but just worth considering as an
alternative, also because, having it dynamic allows us to do more
interesting things later on.
--
Stefano.
Re: [RT] Generator vs Serializer
Posted by Leo Simons <ls...@jicarilla.org>.
Sam Ruby wrote:
> At the moment, gump.document.* take a complete set of knowledge and
> produce a set of artifacts. The best analogy to Cocoon for this would
> be a serializer which terminates a pipeline.
>
> An alternate approach would be to completely flip this. Have the
> equivalent logic drive the acquisition of certain pieces of information,
> which can be processed as it is being received. The best analogy to
> Cocoon for this would be a generator which is at the beginning of a
> pipeline.
Hmm. Sounds like a plan. One comment that has always stuck regarding
cocoon's SAX-based model is that "it is much harder to write components
for it". I don't find gump easy to program for, but I think a large part
of that is because
> not having ANY output until EVERYTHING is done as
> something less than ideal
and if flipping things make that easier that will probably make
everything easier.
I've been thinking about how one does stuff like this (you know,
relatively big architecture flips) incrementally. Seems hard :D. In this
particular case, it seems we need to go from
+----------+ +------------+
| Pipeline | --> | Serializer |
+----------+ +------------+
to
+-----------+ +----------+ +------------+
| Generator | --> | Pipeline | --> | Serializer |
+-----------+ +----------+ +------------+
to
+-----------+ +----------+
| Generator | --> | Pipeline |
+-----------+ +----------+
I took a look around the code and tried to figure out where to start. No
luck so far. I guess I just have to start with small itches ;)
--
cheers,
- Leo Simons
-----------------------------------------------------------------------
Weblog -- http://leosimons.com/
IoC Component Glue -- http://jicarilla.org/
Articles & Opinions -- http://articles.leosimons.com/
-----------------------------------------------------------------------
"We started off trying to set up a small anarchist community, but
people wouldn't obey the rules."
-- Alan Bennett
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: [RT] Generator vs Serializer
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
> For us to get to the point where others are interested in personal
> gumps, we need to make it easier to build profiles which use
> repositories for components that an individual is not interested in
> rebuilding for themselves.
Yeah, I agree, I also think this'd help more Gump 'communities' (not just
the Apache-centric one). One aspect that greatly interests me is 'cascading
Gumps', where one (downstream) Gump downloads some previously Gump'ed jars
from other (upstream) Gumps.
To make general progress in this area, I've introduced three types of
repository for a module i.e. CVS|SVN|Jar, where one 'updates' from these. A
Jar repository update is simply downloading packages from a remote
repository. I've been considering using Depot Ruper for this (a downloader
tool that tries to get the latest version 'intelligently'), but haven't
finished off that wiring.
The other aspect of this (as I see it) is to be able to dynamically override
(or re-write) metadata for a module to replace <cvs with <jar as
appropriate. When I was working more on personal Gumps, and less on main
Gumps, it was a pain in the rear to manually do this.
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: [RT] Generator vs Serializer
Posted by Sam Ruby <ru...@apache.org>.
Adam R. B. Jack wrote:
>
>>An alternate approach would be to completely flip this. Have the
>>equivalent logic drive the acquisition of certain pieces of information,
>>which can be processed as it is being received.
>
> Whilst most of the forrest documenter generates pages for single entities
> (for workspace/module/project/work) there are a significant number of
> statistics/xref pages that rely on all information being available. As such
> we'll need at least two phases.
"classic" gump handles such statistics pages separately too.
>>I'm not much into abstract architectural discussion. I tend to focus on
>>tangible and measurable quanties that matter to real people. In this
>>case, it is clear that Gump is expected to run for a long period of
>>time, and I view not having ANY output until EVERYTHING is done as
>>something less than ideal.
>
> The use case of having 'personal Gumps' (not just relying upon the remote
> servers) has not been greatly exercised. [In the main 'cos I don't have the
> resources to run it locally]. We are moving more and more towards that usage
> though, so I could see how this could start to become important to folks.
I actually think there is considerable value in being able to view live
results in the mult-hour run too, but it is fair to observe that classic
gump "grew" in the other direction - being primarily used for my own
personal builds which I ran multiple times a day, and an occasional
complete resysnc, which I did every couple of weeks.
For us to get to the point where others are interested in personal
gumps, we need to make it easier to build profiles which use
repositories for components that an individual is not interested in
rebuilding for themselves.
- Sam Ruby
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: [RT] Generator vs Serializer
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
> At the moment, gump.document.* take a complete set of knowledge and
> produce a set of artifacts.
Sounds accurate. [I am not familiar w/ Cocoon, so won't comment there.]
> An alternate approach would be to completely flip this. Have the
> equivalent logic drive the acquisition of certain pieces of information,
> which can be processed as it is being received.
I have no objection to this approach being implemented (see next e-mail).
The current design occured out of the developer (me) wanting to use Forrest
[to not get personally bogged down in HTML presentation], and not being
familiar with the Forrest/Cocoon webapp approach. If I'd known then what I'd
known now, I may well have chosen that approach.
Whilst most of the forrest documenter generates pages for single entities
(for workspace/module/project/work) there are a significant number of
statistics/xref pages that rely on all information being available. As such
we'll need at least two phases.
> I'm not much into abstract architectural discussion. I tend to focus on
> tangible and measurable quanties that matter to real people. In this
> case, it is clear that Gump is expected to run for a long period of
> time, and I view not having ANY output until EVERYTHING is done as
> something less than ideal.
The use case of having 'personal Gumps' (not just relying upon the remote
servers) has not been greatly exercised. [In the main 'cos I don't have the
resources to run it locally]. We are moving more and more towards that usage
though, so I could see how this could start to become important to folks.
Given a personal (perhaps debug) run, I can imagine somebody wanting to
monitor success as it goes. No point allowing an N hour test build to
continue if a key component fails early. Right now that is logged, and one
has to read the logs @ debug level to know.
I've thought about scripts and/or GUIs performing the various builds, and I
like the idea of a listener interface for state, and/or perhaps even a
listener interface for various stages (e.g. the trigger you state).
Personally, I'd like the context information to be stored and then presented
to the listener/trigger, so it is available for later phases & isn't lost.
> Producing output as the information becomes available can also
> dramatically reduce working set sizes.
FWIIW: In hindsight (having fixed a bug) .... there is insignificantly
little involved in storing the results tree in memory (a few
states/annotations/file references) compared to the metadata, and the pages
themselves. The 'big leak' was that the document objects (with DOM trees)
were not being freed up, despite the document variable being re-used.
Unlinking trees helped Python play nice, and this is no longer a significant
aspect.
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
[RT] Generator vs Serializer
Posted by Sam Ruby <ru...@apache.org>.
Adam R. B. Jack wrote:
>
> We have gump.document.text, and we could create gump.document.html
> that use cheetah to write it.
Stefano Mazzocchi wrote:
>
> +1 in removal of forrest and go plain XHTML + CSS. But please, let's use
> a velocity-like approach, not a DOM like approach!
I may be reading too much into Stefano's words, but if so, the following
is how I see things.
At the moment, gump.document.* take a complete set of knowledge and
produce a set of artifacts. The best analogy to Cocoon for this would
be a serializer which terminates a pipeline.
An alternate approach would be to completely flip this. Have the
equivalent logic drive the acquisition of certain pieces of information,
which can be processed as it is being received. The best analogy to
Cocoon for this would be a generator which is at the beginning of a
pipeline. Classic gump is closer to this model... after a brief
generation phase, the execution of the resulting script triggers
actions, the output of which is intermixed with some static and some
generated output.
I'm not much into abstract architectural discussion. I tend to focus on
tangible and measurable quanties that matter to real people. In this
case, it is clear that Gump is expected to run for a long period of
time, and I view not having ANY output until EVERYTHING is done as
something less than ideal.
Producing output as the information becomes available can also
dramatically reduce working set sizes.
- Sam Ruby
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump docs solution, makes everyone happy (Re: Gump Thrashing/Spinning...)
Posted by Stefano Mazzocchi <st...@apache.org>.
Sam Ruby wrote:
> Beyond that, I would like to reiterate the point that there is value in
> keeping true to the original design where Gump bootstraps its own
> dependencies.
I agree with that.
--
Stefano.
Re: Gump docs solution, makes everyone happy (Re: Gump Thrashing/Spinning...)
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Sam Ruby wrote:
> Nicola Ken Barozzi wrote:
>
...
>> Maybe my comment got lost... generate html and Forrest can skin that too.
>>
>> In other words, Forrest can skin an html site.
>>
>> So, if you make Cheetah output plain html you can see the site
>> natively, or decide to have Forrest skin over it and publish it or use
>> a live Forrest.
>>
>> I'm -1 on removing Forrest for output, as it takes away the same
>> visual style of the site.
>
> That's darn near a circular definition.
I mean that since I want to keep the site done with Forrest, I would
also like to have the other info done that way.
> To demonstrate: what's wrong
> with the notion of switching the site to Anakia, which is stable, builds
> consistently with Gump, and powers www.apache.org, jakarta.apache.org,
> and a number of other sites?
>
> Now realize that I am *NOT* proposing Anakia. What I am proposing is
> that the ability to view a site as it is being produced is a very
> valuable thing to have, and an important consideration both for a
> machine which is a shared resource and for any hope of there ever being
> personal usage of gump.
Errr, did I say that this is not ok?
Replay:
">> So, if you make Cheetah output plain html you can see the site
>> natively, or decide to have Forrest skin over it and publish it or
>> use a live Forrest.
"
I am trying to say that I am *+1* to outputting html, but also that
Forrest can layer *on top* of that to make a skinned version *if* the
user wants to. Note that this does not prevent you from putting a css in
the html that Forrest can skin, so that even the non-forrest-skinned
version can have nice looks.
Is this clear?
> Beyond that, I would like to reiterate the point that there is value in
> keeping true to the original design where Gump bootstraps its own
> dependencies.
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump docs solution, makes everyone happy (Re: Gump Thrashing/Spinning...)
Posted by Nick Chalko <ni...@chalko.com>.
Adam R. B. Jack wrote:<snip/>
>You are preaching to the choir. We've agonized over installing a Maven drop
>to build Maven projects. Unfortunately, we can't rely solely upon a (higher
>dependency) Gumped application for output, 'cos they (e.g Maven and
>Forrest/Cocoon) build so infrequently. [Good intentions can't force
>consistent builds.] The case of Maven is more persuasive than a
>documentation tool (of which we could pick) because communities have picked
>Maven.
>
>
Here is another example why we should try to build with the last good
build of dependencies.
R,
Nick
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump docs solution, makes everyone happy (Re: Gump Thrashing/Spinning...)
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
> Now realize that I am *NOT* proposing Anakia. What I am proposing is
> that the ability to view a site as it is being produced is a very
> valuable thing to have, and an important consideration both for a
> machine which is a shared resource and for any hope of there ever being
> personal usage of gump.
Sam, you seem to care more about timeliness of content than tooling, and I
can respect that & I think it can be accomodated w/o ruling out forrest. For
example, what is wrong with writting HTML and/or XDOCS (whatever) as things
go along? If we wrote xdocs to a webapp the cocoon/forrest webapp could
detect the file change and rerender. I see nothing about timeliness that
dictates format.
BTW: We could do this today with gump.document.forrest -- calling the
documentProject and documentModule methods with context. Not much in this
code (other than stats/xref) cares when you call it. I'm not neccessarily
proposing that, just stating we aren't so far off.
> Beyond that, I would like to reiterate the point that there is value in
> keeping true to the original design where Gump bootstraps its own
> dependencies.
You are preaching to the choir. We've agonized over installing a Maven drop
to build Maven projects. Unfortunately, we can't rely solely upon a (higher
dependency) Gumped application for output, 'cos they (e.g Maven and
Forrest/Cocoon) build so infrequently. [Good intentions can't force
consistent builds.] The case of Maven is more persuasive than a
documentation tool (of which we could pick) because communities have picked
Maven.
I'd love to developer a solution where we use the Gumped solution (if
available) but fallback to a packaged/installed solution when not.
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump docs solution, makes everyone happy (Re: Gump Thrashing/Spinning...)
Posted by Sam Ruby <ru...@apache.org>.
Nicola Ken Barozzi wrote:
> Adam R. B. Jack wrote:
>
>>> I'll certainly guilty of being away for a while, but
>>> gump.document.forrest is not a small thing, and to my eyes, not entirely
>>> obvious.
>
> ...
> > Also, if we want both xdoc and html output we'd need to set
>
>> of tempaltes (with code in) which isn't nice.
>
> Maybe my comment got lost... generate html and Forrest can skin that too.
>
> In other words, Forrest can skin an html site.
>
> So, if you make Cheetah output plain html you can see the site natively,
> or decide to have Forrest skin over it and publish it or use a live
> Forrest.
>
> I'm -1 on removing Forrest for output, as it takes away the same visual
> style of the site.
That's darn near a circular definition. To demonstrate: what's wrong
with the notion of switching the site to Anakia, which is stable, builds
consistently with Gump, and powers www.apache.org, jakarta.apache.org,
and a number of other sites?
Now realize that I am *NOT* proposing Anakia. What I am proposing is
that the ability to view a site as it is being produced is a very
valuable thing to have, and an important consideration both for a
machine which is a shared resource and for any hope of there ever being
personal usage of gump.
Beyond that, I would like to reiterate the point that there is value in
keeping true to the original design where Gump bootstraps its own
dependencies.
- Sam Ruby
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump docs solution, makes everyone happy (Re: Gump Thrashing/Spinning...)
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
> > Also, if we want both xdoc and html output we'd need to set
> > of tempaltes (with code in) which isn't nice.
>
> Maybe my comment got lost... generate html and Forrest can skin that too.
>
> In other words, Forrest can skin an html site.
I heard it, but I think I mentally filtered it somewhat, wondering if it was
some sort of unholy compromise. Can you give more details on how this works,
and the pros/cons of using HTML not xdocs? Does one simply write normal (but
parsable) HTML and Forrest parse/processes it? Can we use the webapp
approach with this output?
BTW: With Forrest moving to xhtml eventually, does us moving to HTML help
us?
> So, if you make Cheetah output plain html you can see the site natively,
> or decide to have Forrest skin over it and publish it or use a live
Forrest.
I don't have Stefano's dislike of DOM verses template generation. Perhaps
because I wrote the code, perhaps because I am game to trade cycles/memory
for easier coding. I know the code seems to be complex (at first) but I'll
explain (see mail, to follow, on 'Documentation') that it really isn't. I
want to see a Cheetah based approach (it seems interesting & the Pythonic
defacto standard) but I honestly have reservations about (1) it being easier
(2) it being more manageable. Still, hopefully a good design (with Cheetah
approach in mind) ought let us know in advance if this is the case.
> I'm -1 on removing Forrest for output, as it takes away the same visual
> style of the site.
I've been frustrated with forrest spinning (it did it again last night for
my machine) and builds working, but failing to output documentation. I
proposed moving away from Forrest for this reason, no other. I think it has
emerged that forrest is more of a stumbling block for others -- especially
newcomers -- and it too much to expect/install (even thought it builds out
of the box, and runs first time) for starters. As such, I am definately -1
to removing it completely, and +1 to having a simple HTML solution for
starters. See mail (to follow) on 'Documentation'.
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Gump docs solution, makes everyone happy (Re: Gump Thrashing/Spinning...)
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Adam R. B. Jack wrote:
>>I'll certainly guilty of being away for a while, but
>>gump.document.forrest is not a small thing, and to my eyes, not entirely
>>obvious.
...
> Also, if we want both xdoc and html output we'd need to set
> of tempaltes (with code in) which isn't nice.
Maybe my comment got lost... generate html and Forrest can skin that too.
In other words, Forrest can skin an html site.
So, if you make Cheetah output plain html you can see the site natively,
or decide to have Forrest skin over it and publish it or use a live Forrest.
I'm -1 on removing Forrest for output, as it takes away the same visual
style of the site.
IMHO making Forrest skin the html is the most reasonable way to go.
> Still, I think forrest is a sticking point. I'd like a pure Python solution,
> and I suspect Cheetah is the best way to go.
Cheetah is IMHO the way to go for the basic html rendering.
> BTW: I'm eager to see some graphics. If SVG can be written (in XML, viua
> Python) can Forrest convert these to images (using Batik or somethough?)
> I'll happily continue with Forrest if that is a good way to get images
> rendered.
It's there since some time now.
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump Thrashing/Spinning...
Posted by Nick Chalko <ni...@chalko.com>.
Adam R. B. Jack wrote:
>BTW: I'm eager to see some graphics. If SVG can be written (in XML, viua
>Python) can Forrest convert these to images (using Batik or somethough?)
>I'll happily continue with Forrest if that is a good way to get images
>rendered.
>
>
Yes
Just put foo.svg and then access it as foo.png
For example
http://antworks.sourceforge.net/importer/images/project-logo.png
Comes from
<?xml version="1.0" encoding="Windows-1252"?>
<svg xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 340 60"
height="60" width="340">
<title>antworks-importer logo</title>
<defs><linearGradient y2="1" x2="0" y1="0" x1="0" id="gradient"><stop
offset="0" style="stop-color:white"/>
<stop offset="1" style="stop-color:lightgreen"/></linearGradient>
<filter filterUnits="objectBoundingBox" id="shadowFilter">
<feGaussianBlur result="blur" stdDeviation="2 2" in="SourceAlpha"/>
<feOffset result="offsetBlur" dy="4" dx="4" in="blur"/>
<feComposite operator="over" in2="offsetBlur"
in="SourceGraphic"/></filter></defs>
<g fill="url(#gradient)" filter="url(#shadowFilter)">
<text style="font-size:32pt; font-family:Verdana ; text-anchor: middle"
y="50%" x="45%">Antworks Importer</text>
<text style="font-size:10pt; font-family:Arial ; text-anchor: middle"
y="80%" x="55%">autdownload and import ant build.xml</text></g>
</svg>
Is generated by the forrest.antlet
http://antworks.sourceforge.net/antlets/forrest/xbuild-doc.html
from a module.xml
R,
Nick
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump Thrashing/Spinning...
Posted by "Adam R. B. Jack" <aj...@trysybase.com>.
> I'll certainly guilty of being away for a while, but
> gump.document.forrest is not a small thing, and to my eyes, not entirely
> obvious.
It is really just a lot of repetative simple code, building simple xdoc
pieces. It it's pretty, but it isn't that bad.
We have gump.document.text, and we could create gump.document.html that use
cheetah to write it.
I looked at Cheetah and liked the ability to create templates as classes,
and sub-class (perhaps aggregate) such, but when I thought about it I
couldn't really find much real different from what was going on with
gump.document.forrest. Basically Cheetah requires code within the template
to map memory into locations, and I didn't find it hughly different than
what is there. Also, if we want both xdoc and html output we'd need to set
of tempaltes (with code in) which isn't nice.
Still, I think forrest is a sticking point. I'd like a pure Python solution,
and I suspect Cheetah is the best way to go.
BTW: I'm eager to see some graphics. If SVG can be written (in XML, viua
Python) can Forrest convert these to images (using Batik or somethough?)
I'll happily continue with Forrest if that is a good way to get images
rendered.
> A design point for the original gump is that output was viewable as it
> was produced. Is this the case for gumppy w/ forrest?
No, everything is stored in files with memory context. With
gump.document.forrest we write a bunch of xdocs in one pass (before we call
forrest).
regards,
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump Thrashing/Spinning...
Posted by Sam Ruby <ru...@apache.org>.
Nick Chalko wrote:
> Stefano Mazzocchi wrote:
>
>> \
>>
>>> .
>>
>> Ok, I think that reducing complexity is critical for wider adoption.
>>
>> +1 in removal of forrest and go plain XHTML + CSS. But please, let's
>> use a velocity-like approach, not a DOM like approach!
>
> I am not sure how removing forrest reduces complexity. It just means we
> have to maintaining the template / site code ourself.
I'll certainly guilty of being away for a while, but
gump.document.forrest is not a small thing, and to my eyes, not entirely
obvious.
A design point for the original gump is that output was viewable as it
was produced. Is this the case for gumppy w/ forrest?
- Sam Ruby
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump Thrashing/Spinning...
Posted by Nick Chalko <ni...@chalko.com>.
Stefano Mazzocchi wrote:
> \
>
>> .
>
>
> Ok, I think that reducing complexity is critical for wider adoption.
>
> +1 in removal of forrest and go plain XHTML + CSS. But please, let's
> use a velocity-like approach, not a DOM like approach!
I am not sure how removing forrest reduces complexity.
It just means we have to maintaining the template / site code ourself.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump Thrashing/Spinning...
Posted by Stefano Mazzocchi <st...@apache.org>.
Adam Jack wrote:
> Unfortunately we are stuck with a recent CVS HEAD, if not latest, due to
> some features in the skin. We can't go back to a release.
>
> FWIIW: The release we have has been working for the last month or so,
> something just changed and dorked it.
Ok, I think that reducing complexity is critical for wider adoption.
+1 in removal of forrest and go plain XHTML + CSS. But please, let's use
a velocity-like approach, not a DOM like approach!
--
Stefano.
Re: Gump Thrashing/Spinning...
Posted by Adam Jack <aj...@TrySybase.com>.
Unfortunately we are stuck with a recent CVS HEAD, if not latest, due to
some features in the skin. We can't go back to a release.
FWIIW: The release we have has been working for the last month or so,
something just changed and dorked it.
regards
Adam
----- Original Message -----
From: "Michael Davey" <Mi...@coderage.org>
To: "Gump code and data" <ge...@gump.apache.org>
Sent: Thursday, March 25, 2004 4:01 PM
Subject: Re: Gump Thrashing/Spinning...
> Adam Jack wrote:
>
> > [snip]
> >
> >I think it is forrest that is where we are getting stuck right now.
> >Now sure why, but it is locking up. So, if we want forest, we have to
figure
> >out how to get inside that problem.
> >
> >
> Which version are you using? Probably coincidence, but I recently
> stopped using
> CVS HEAD because cocoon would hang during the Forrest build-site stage.
> Version 0.51 of Forrest is quite happy building my site from exactly the
> same
> xdocs source.
>
> --
> Michael
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
> For additional commands, e-mail: general-help@gump.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump Thrashing/Spinning...
Posted by Michael Davey <Mi...@coderage.org>.
Adam Jack wrote:
> [snip]
>
>I think it is forrest that is where we are getting stuck right now.
>Now sure why, but it is locking up. So, if we want forest, we have to figure
>out how to get inside that problem.
>
>
Which version are you using? Probably coincidence, but I recently
stopped using
CVS HEAD because cocoon would hang during the Forrest build-site stage.
Version 0.51 of Forrest is quite happy building my site from exactly the
same
xdocs source.
--
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump Thrashing/Spinning...
Posted by Adam Jack <aj...@TrySybase.com>.
> > 1) use forrest as a dynamic application
First, what do you mean by this, please? For those of us who don't know,
could somebody elaborate?
Second, I think it is forrest that is where we are getting stuck right now.
Now sure why, but it is locking up. So, if we want forest, we have to figure
out how to get inside that problem.
regards
Adam
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump Thrashing/Spinning...
Posted by Nick Chalko <ni...@chalko.com>.
Stefano Mazzocchi wrote:
> Adam Jack wrote:
>
>
> Two possible solutions here:
>
> 1) use forrest as a dynamic application
> 2) have HTML generate by python
>
> I would go for 1) since it would keep us the ability to do dynamic
> stuff like metadata manipulation.
>
> I volunteer to setup 1)
>
+1 for dynamic forrest.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org
Re: Gump Thrashing/Spinning...
Posted by Stefano Mazzocchi <st...@apache.org>.
Adam Jack wrote:
> I've been trying to run some quick jobs (a check.py not an integrate.py) to
> try to get insight into this problem. I'm finding (at least on my machine)
> that forrest is growing to huge size, and getting bogged down in swapping.
>
> I can't say this with certainty, but I feel that both Python and Java
> degrade very poorly when swapping, not just because they do so much dynamic
> memory management. I suspect any background thread for garbage collection is
> getting starved. I don't know if this is true, or even if it matters, but we
> are getting to a point of resoruce shortage, to the point of outage.
>
> I wonder if Python (Gumpy) and Java (Forrest ) are fighting with each other
> for resources. Both are hogs, but I am not sure if being a memory hog is
> cause or effect. I don't know if growth is occuring 'cos we finally added
> the last straw to the camel's back, or if we have some wierd loop bug. I
> suspect forrest is getting stuck on a certain page, but I have no insight
> into what is causing this -- nor if the cause is somehow Gumpy. Doing a much
> smaller Gump run, hence smaller xdocs set, works w/o problem.
>
> When doing a test run here (without even doign the build parts) I see Gumpy
> (the python instance) growing to 136M! I have no clue as to why it would do
> that! I wish I could get inside the Python instance to understand where the
> memory is, or if it is being leaked. Maybe this is just crowding out
> forrest.
>
> Whatever the cause, I am really starting to get 'done' with forrest. I
> support it's use, I introduced it & have dealt with the issues and built
> workarounds from day one, but it is hard work w/ no fun. I feel it (through
> Gumpy trying to dynamically create xdocs) has been the weak/slow link in
> Gumpy for a while.
>
> I suspect even the forrest developers here might agree that I took this too
> far. Forrest is great for sites, but the gump outputs are getting huge and
> not really benefiting from Forrest skins, etc. Maybe I am wrong, maybe I
> need to approach the forrest developers (mail to their team) to see what
> they think on this.
>
> I wonder if we ought consider replacing Forrest with a pure Python HTML
> producer. As above, I can't prove that forrest is the problem, but a pure
> Python solution might just halve the unknowns.
Two possible solutions here:
1) use forrest as a dynamic application
2) have HTML generate by python
I would go for 1) since it would keep us the ability to do dynamic stuff
like metadata manipulation.
I volunteer to setup 1)
--
Stefano.
Re: Gump Thrashing/Spinning...
Posted by "matthew.hawthorne" <ma...@apache.org>.
> I wonder if we ought consider replacing Forrest with a pure Python HTML
> producer. As above, I can't prove that forrest is the problem, but a pure
> Python solution might just halve the unknowns.
I've been using Python to generate HTML from XML via XSL for a few weeks
now using 4suite. I'm fairly new to Python so I wasn't really sure what
my options were for XSL. It's been very good so far, speed wise.
So, maybe it's an option.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org