Pipeline recursion (sort of)?

["Hunsberger, Peter" <Pe...@STJUDE.ORG>]

I've been starting to generalize our systems architecture and I realized
that there is one generalization that might be problematic with Cocoon.
Basically, we allow a hierarchy of metadata.  For example (use fixed
fonts):

    system
       |
       V
  application
       |
       V
    service
       |
       V
   division
       |
       V
   department
       |
       V
    section
       |
       V
   individual

At each level it is possible to override the metadata of the previous
level. We currently support two (sort of three) layers of metadata with
hard coded generators for each layer and a subsequent XSLT that merges
the results. The generalization I'm after would allow the number of
levels to be dynamically determined for each individual.  

The simple minded implementation just runs a single generator that
handles all the dynamics and spits out the merged results. The big issue
is caching.  As you move down the hierarchy each level of metadata has
to be cached for a smaller population, but the odds of it changing are
smaller and it can likely be cached for a longer period of time.
Basically, I want each level of metadata to have it's own cache key and
cache validity. 

I think I might be able to do this with flow script and
"processPipelineTo"?  I'm guessing I'd be able to just keep looping on
the pipeline calls (a previous pipeline would supply the meta-metadata
that tells me what metadata is required dynamically)? Each invocation
would basically pick up a variable from the previous invocation and add
it's results in some predictable pattern. If I subsequently re-invoke a
pipeline with "processPipelineTo" is there any reason the normal cache
checking wouldn't work?

The biggest problem I see with this approach is the cacheability of the
overall results (which we would really need).  Here I think I'd be able
to use some kind of composite cache validity, basically picking up the
cache validity from each step of the "processPipelineTo" calls and using
them in some kind of container cache validity checker that would
determine if the overall pipeline was still valid (basically a identity
transformer that does nothing but generate a cache key and cache
validity on the overall pipeline results). How the heck would I pick up
the cache validity objects from the processPipelineTo steps for the
overall composite validity? 

Any of this make sense, or am I wondering off into a world of
generalization that Cocoon is just not suited for?

Peter Hunsberger