You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@turbine.apache.org by Gareth Coltman <ga...@majorband.co.uk> on 2002/02/01 17:34:33 UTC
RE: Thinking about 'multiple' pipeline (was Re: Possible patch to allowfiles to be streamed back from action)
James,
>
> <pipeline>
> <name>TurbineClassicPipeline</name>
> <valves>
> <valve
> className="org.jamestaylor.myapp.pipeline.DetermineTargetValve"/>
> <valve className="org.jamestaylor.myapp.pipeline.RunModulesValve"/>
> <select
> className="org.jamestaylor.myapp.pipeline.TargetExtensionSelector">
> <when test="vm">
> <valve
> className="org.jamestaylor.myapp.pipeline.DirectVelocityRendererValve"/>
> </when>
> <when test="jsp">
> <valve
> className="org.jamestaylor.myapp.pipeline.JSPRendererValve"/>
> </when>
> <when test="swt">
> <valve
> className="org.jamestaylor.myapp.pipeline.JGenRendererValve"/>
> </when>
> </select>
> </valves>
> </pipeline>
I think this is an excellent approach. Only problem I can see is what if
there is no extension in the request. What if I want the result of a form
submission to my FDF. I guess I could just put something like:
action="$link.setAction(...).setPage(".FDF")"
or something like that.
> This gives users a ton of flexibility. We could implement some standard
> selectors, for example ones that look at request parameters or objects
> stored in RunData (however that looks when Jason finishes with it). The
> base classes for selectors could be more intelligent, to allow globbed
> or regexp matches.
yes. yes. yes.
> In Gareth's case, this would allow him a few ways to accomplish his
> goal. He could simply leave out any target valve for certain extensions,
No I don't like this, I would prefer to do all rendering in a target valve.
This makes more sense in the design. I guess I just need to make sure that
the relevant data in the the session.
> Or (the one I prefer) he could implement a valve for
> rendering using the FDF sdk, and use a selector to ensure that that
> valve is run for the relevent targets.
>
Yes I prefer this but its difficult to see how it would work:
ACTION:
1.Parse Form data
2.Put it in session?
VALVE:
1. Get data from session
2. Render appropriate FDF File
????
Not really sure about this approach. At the moment the result of the action
is a byte[] (or String actually) which represents the FDF. I could store
just this output in the session. Then print it to the output stream in the
valve. Seems like a weird way to pass data around though.
What to you think?
Gareth
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: torque caching and JCS as a turbine service (credit
cardprocessing)
Posted by Jon Scott Stevens <jo...@latchkey.com>.
on 2/4/02 5:39 PM, "Aaron Smuts" <aa...@verizon.net> wrote:
> I wonder if caching should be above or on top of Torque rather than
> within.
I'm not really sure that putting it on top of Torque will achieve the
desired effect of transparent caching. Something like that would need to
come from within.
That said, it should be easily disabled if needed. So, the caching methods
should always call: "isCacheEnabled()" or something like that at the top of
their execution.
-jon
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: torque caching and JCS as a turbine service (credit cardprocessing)
Posted by Aaron Smuts <aa...@verizon.net>.
> -----Original Message-----
> From: Jon Scott Stevens [mailto:jon@latchkey.com]
> Sent: Monday, February 04, 2002 12:46 PM
> To: turbine-dev
> Subject: Re: torque caching and JCS as a turbine service (credit
> cardprocessing)
>
> on 2/3/02 7:51 PM, "Aaron Smuts" <aa...@verizon.net> wrote:
>
> > Back to the point. I could have the managers use Torque for data
access
> > and the managers could still handle the caching, but it might be
nice to
> > have torque handle some of this on its own. The complexity of
handling
> > group dependencies on items matching certain parameters might be a
mess,
> > but for item level stuff it would be nice. Sort of like Javlin
(sp?)
> > and an appserver all in one.
> >
> > Where might caching fit into torque?
> >
> > What about JCS as a turbine service?
> >
> > Aaron
>
> Just some thoughts...
>
> I need caching in Torque as well...I suspect that all of the
doSelect(),
> doUpdate(), doDelete() methods in the Peers would need to be made
> intelligent about caching. We would also want the field level methods
(the
> get/setters) and the save() method in the BO to help with invaliding
and
> updating the cache. You also would need to add some sort of dependency
> tracking into the objects which are related to each other through
joins in
> the database so that when one changes, the other changes as well. This
> part
> can turn into a logistical nightmare if you have several levels of
joins.
>
I wonder if caching should be above or on top of Torque rather than
within.
> Of course all of this would need to be able to be turned off at a
moments
> notice as well...however, if it is done right, it should be
unnoticeable
> when caching is turned on because the invalidation/updating will be so
> efficient and work so well, that you won't even know that an object
came
> from the database or from the cache...
>
> -jon
>
>
> --
> To unsubscribe, e-mail: <mailto:turbine-dev-
> unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:turbine-dev-
> help@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: torque caching and JCS as a turbine service (credit card
processing)
Posted by Jon Scott Stevens <jo...@latchkey.com>.
on 2/3/02 7:51 PM, "Aaron Smuts" <aa...@verizon.net> wrote:
> Back to the point. I could have the managers use Torque for data access
> and the managers could still handle the caching, but it might be nice to
> have torque handle some of this on its own. The complexity of handling
> group dependencies on items matching certain parameters might be a mess,
> but for item level stuff it would be nice. Sort of like Javlin (sp?)
> and an appserver all in one.
>
> Where might caching fit into torque?
>
> What about JCS as a turbine service?
>
> Aaron
Just some thoughts...
I need caching in Torque as well...I suspect that all of the doSelect(),
doUpdate(), doDelete() methods in the Peers would need to be made
intelligent about caching. We would also want the field level methods (the
get/setters) and the save() method in the BO to help with invaliding and
updating the cache. You also would need to add some sort of dependency
tracking into the objects which are related to each other through joins in
the database so that when one changes, the other changes as well. This part
can turn into a logistical nightmare if you have several levels of joins.
Of course all of this would need to be able to be turned off at a moments
notice as well...however, if it is done right, it should be unnoticeable
when caching is turned on because the invalidation/updating will be so
efficient and work so well, that you won't even know that an object came
from the database or from the cache...
-jon
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
torque caching and JCS as a turbine service (credit card processing)
Posted by Aaron Smuts <aa...@verizon.net>.
I'd like to get some ideas on how caching might be integrated into
Turbine and Torque.
At my job I've implemented a pattern where data access goes through
managers that return value objects, except on large result sets (they
are cached where efficient and sorted using a comparator sorter on
scaled down value objects or on ArrayLists of arrays).
The managers return simple lists for drop downs and the value objects
for detail pages. The caching is done at the manager level. Data comes
into our application via remote check processing centers and is parsed
by standalone parsers. Credit card and ACH transactions processed in
batch more also run in standalone mode. All the processes access the
data via the same managers and all communicate invalidations through the
cache. Conglomerate data is invalidated by adding the summary data
cache name to the list of caches to notify of the changes for items.
The local and mid-tier caching system keeps the 12 or so machines in
synch. It works nicely. Of course some data access is not cached since
it needs to be 100% accurate. . . .
(Off the subject but a Visa Gateway server certified by Vital would be a
fantastic jakarta project. If you ever needed to handle credit card
processing, you could turn to an open source solution! I'm sure I could
use it in future jobs. Right now you have to buy something less
complicated than JCS that costs at least 20k to send files to the credit
card processing network. I'm all for starting the project if anyone is
interested. You can build a standalone server that takes basic input,
formats a file and sends it to a configured provider. There are some
tricky error handling issues, but it would be an extremely useful
project, but it may put some people out of business.)
Back to the point. I could have the managers use Torque for data access
and the managers could still handle the caching, but it might be nice to
have torque handle some of this on its own. The complexity of handling
group dependencies on items matching certain parameters might be a mess,
but for item level stuff it would be nice. Sort of like Javlin (sp?)
and an appserver all in one.
Where might caching fit into torque?
What about JCS as a turbine service?
Aaron
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Thinking about 'multiple' pipeline
Posted by James Taylor <jt...@4lane.com>.
> Definitely cool, but I would like to leave the conditional aspects as
> optional so that in the case where you have an application that is using a
> single view type is optimized.
I agree, and a solution like this gives total flexibility. If you want
to just have a straight pipeline you don't use any conditional valves.
If you want more complicated conditional selection you can use the
conditional valves (and selector valves once built) to accomplish almost
anything. They can be nested without theoretical limit.
> I think your valves would also be very
> valuable for people migrating apps from jsp to velocity (no one would ever
> go the other way!)
Or using both at the same time. Which I agree is unlikely, but there are
other things that fit into this model that people might want to be able
to use concurrently, like vm, image generation with batik, pdf with fop
(or the fdf sdk that several people seem to need, whatever it is), flash
content with JGen.
> Does the pipeline you whipped up make jsp work with the classic model? That
> would be very cool if it does!
Yup! Well, no layouts, but the standard action - module - template
chaining model should work. The archive I sent you privately has a
config in src/rttest/conf/turbine-testapp-pipeline.xml that should
support the classic model for vm, jsp, and set with selection based on
the extension, although I believe it leaves out RunModulesValve for jsp
since I was emulating the other jsp pipeline.
> John and I also briefly discussed a top-level selector, as I agreed with him
> when said he didn't like the idea of being able to select a pipeline with
> parameters. It would also be nice to be able to avoid the selector as well
> for say a pure velocity app so we can make things as optimal as possible.
Totally. I think doing the selectors with valves is the way to go since
you get full flexibility. You can have a selector at the top (top level
pipeline with one selector valve containing other pipelines that get
executed conditionally), Just a standard pipeline (no conditional or
selection valves at all), or any more complicated flow.
The mapper makes the easy because we can pass arbitrary implementation
specific parameters (in the form of bean properties) to the valves for
configuration. Thus people can subclass the BaseConditionalValve (and
BaseSelectorValve or whatever we end up with for that) and configure it
however they want in the pipeline xml file.
-- jt
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Thinking about 'multiple' pipeline
Posted by Jason van Zyl <jv...@zenplex.com>.
On 2/1/02 7:59 PM, "James Taylor" <jt...@4lane.com> wrote:
> I put together an implementation of some of the ideas I discussed
> earlier, in the form of and abstract BaseConditionValve and two concrete
> subclasses (request and extension based) that allow a set of valves to
> be included or excluded based on conditions (implementation defined).
> The mapper made this very easy!
>
>
> I'm really happy with the way this turned out. It seems pretty elegant
> considering no changes to existing pipeline classes were required.
> Thoughts?
Definitely cool, but I would like to leave the conditional aspects as
optional so that in the case where you have an application that is using a
single view type is optimized. I think your valves would also be very
valuable for people migrating apps from jsp to velocity (no one would ever
go the other way!)
Does the pipeline you whipped up make jsp work with the classic model? That
would be very cool if it does!
John and I also briefly discussed a top-level selector, as I agreed with him
when said he didn't like the idea of being able to select a pipeline with
parameters. It would also be nice to be able to avoid the selector as well
for say a pure velocity app so we can make things as optimal as possible.
> Thanks,
> James
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
--
jvz.
Jason van Zyl
http://tambora.zenplex.org
http://jakarta.apache.org/turbine
http://jakarta.apache.org/velocity
http://jakarta.apache.org/alexandria
http://jakarta.apache.org/commons
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Thinking about 'multiple' pipeline
Posted by James Taylor <jt...@4lane.com>.
I put together an implementation of some of the ideas I discussed
earlier, in the form of and abstract BaseConditionValve and two concrete
subclasses (request and extension based) that allow a set of valves to
be included or excluded based on conditions (implementation defined).
The mapper made this very easy!
The condition valve essentially wraps a pipeline. When invoked it runs
the doTest method (which subclasses must implement) and if that returns
true in invokes the set of valves it contains, before invokeing its next
sibling.
An implementation that emulates a switch rather than an if should be
easy to put together along the same lines.
I changed the 'classic' pipeline to support vm and jsp, and added a test
to the TurbineSerlvetTest that tests the jsp page Index.jsp, everything
works great. The pipeline config now looks like this:
<pipeline>
<name>TurbineClassicPipeline</name>
<valves>
<valve className="org.apache.turbine.pipeline.DetermineActionValve"/>
<valve className="org.apache.turbine.pipeline.DetermineTargetValve"/>
<valve className="org.apache.turbine.pipeline.DefaultSessionTimeoutValve"/>
<valve className="org.apache.turbine.pipeline.DefaultLoginValve"/>
<valve className="org.apache.turbine.pipeline.DefaultSessionValidationValve"/>
<valve className="org.apache.turbine.pipeline.DefaultACLCreationValve"/>
<valve className="org.apache.turbine.pipeline.DefaultActionValve"/>
<valve className="org.apache.turbine.pipeline.RunModulesValve"/>
<valve className="org.apache.turbine.pipeline.TargetExtensionConditionalValve">
<extension>.vm</extension>
<valves>
<valve className="org.apache.turbine.pipeline.DefaultTargetValve"/>
</valves>
</valve>
<valve className="org.apache.turbine.pipeline.TargetExtensionConditionalValve">
<extension>.jsp</extension>
<valves>
<valve className="org.apache.turbine.pipeline.JspRendererValve"/>
</valves>
</valve>
</valves>
</pipeline>
I'm really happy with the way this turned out. It seems pretty elegant
considering no changes to existing pipeline classes were required.
Thoughts?
Thanks,
James
RE: Thinking about 'multiple' pipeline (was Re: Possible patch to
allowfiles to be streamed back from action)
Posted by James Taylor <jt...@4lane.com>.
> I think this is an excellent approach. Only problem I can see is what if
> there is no extension in the request. What if I want the result of a form
> submission to my FDF. I guess I could just put something like:
>
> action="$link.setAction(...).setPage(".FDF")"
>
> or something like that.
No, you would just use a different implementation of selector, or write
your own which parses some relevent parameter from somewhere and decides
what valve to use based on it. In cocoon the base selectors are browser,
header, host, request parameter, request attribute, session attribute.
We could implement similar selectors based on more turbine relevent
stuff.
Cocoon matchers:
http://cvs.apache.org/viewcvs.cgi/xml-cocoon2/src/java/org/apache/cocoon/matching/
Cocoon selectors:
http://cvs.apache.org/viewcvs.cgi/xml-cocoon2/src/java/org/apache/cocoon/selection/
>
> > This gives users a ton of flexibility. We could implement some standard
> > selectors, for example ones that look at request parameters or objects
> > stored in RunData (however that looks when Jason finishes with it). The
> > base classes for selectors could be more intelligent, to allow globbed
> > or regexp matches.
>
> yes. yes. yes.
>
> > In Gareth's case, this would allow him a few ways to accomplish his
> > goal. He could simply leave out any target valve for certain extensions,
>
> No I don't like this, I would prefer to do all rendering in a target valve.
> This makes more sense in the design. I guess I just need to make sure that
> the relevant data in the the session.
>
> > Or (the one I prefer) he could implement a valve for
> > rendering using the FDF sdk, and use a selector to ensure that that
> > valve is run for the relevent targets.
> >
>
> Yes I prefer this but its difficult to see how it would work:
>
> ACTION:
>
> 1.Parse Form data
> 2.Put it in session?
>
> VALVE:
>
> 1. Get data from session
> 2. Render appropriate FDF File
This is the one I like since it follows the current turbine pattern.
Actions and Modules build contexts, renderer render views. In this case
you don't have a template at all, but you still have an FDFRendererValve
or whatever.
> ????
>
> Not really sure about this approach. At the moment the result of the action
> is a byte[] (or String actually) which represents the FDF. I could store
> just this output in the session. Then print it to the output stream in the
> valve. Seems like a weird way to pass data around though.
>
> What to you think?
I think you have a special case where its going to be hard to make the
FDF sdk fit cleanly into the turbine model. However if we implement
something like what is described above, it would give you enough
flexibility to accomplish what you want in several ways, even if none of
them is ideal.
Thanks,
James
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Thinking about 'multiple' pipeline (was Re: Possible patch to allowfiles to be streamed back from action)
Posted by Daniel Rall <dl...@finemaltcoding.com>.
"Gareth Coltman" <ga...@majorband.co.uk> writes:
> So I once again suggest that we modify the runData. This seems to be the
> only sensible way of sending data from valve to valve.
Using a RunData implementation is the Right Thing to do in this
situation. You can always subclass RunData, then check in your custom
Valve implementations to see if you have an specific type.
Dan
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Thinking about 'multiple' pipeline (was Re: Possible patch to allowfiles to be streamed back from action)
Posted by Gareth Coltman <ga...@majorband.co.uk>.
OK, I've thought more about the problem.
What we need is a way to send data from the action valve to the target
value.
The session (which was my first suggestion) is a terrible work around. We
should never rely on a session being there in the pipeline.
So I once again suggest that we modify the runData. This seems to be the
only sensible way of sending data from valve to valve.
The idea is :
In your action, you create some raw data (pdf, images). You then call
data.setRawData(data, contentType)
This will create a reference to the raw data in the runData and also call
data.setContentType().
Then you set up a new target (using James suggested extension based
configuration) which will do the follwing:
data.getResponse().setContentType(data.getContentType);
data.getResponse().getOutputStream().write(data.getRawData());
Something like that anyway!
OK - this adds another property to the runData but I think that it does
provide more flexibility.
Gareth
> -----Original Message-----
> From: Gareth Coltman [mailto:gareth_coltman@majorband.co.uk]
> Sent: 01 February 2002 16:35
> To: Turbine Developers List
> Subject: RE: Thinking about 'multiple' pipeline (was Re: Possible patch
> to allowfiles to be streamed back from action)
>
>
> James,
>
> >
> > <pipeline>
> > <name>TurbineClassicPipeline</name>
> > <valves>
> > <valve
> > className="org.jamestaylor.myapp.pipeline.DetermineTargetValve"/>
> > <valve className="org.jamestaylor.myapp.pipeline.RunModulesValve"/>
> > <select
> > className="org.jamestaylor.myapp.pipeline.TargetExtensionSelector">
> > <when test="vm">
> > <valve
> > className="org.jamestaylor.myapp.pipeline.DirectVelocityRendererValve"/>
> > </when>
> > <when test="jsp">
> > <valve
> > className="org.jamestaylor.myapp.pipeline.JSPRendererValve"/>
> > </when>
> > <when test="swt">
> > <valve
> > className="org.jamestaylor.myapp.pipeline.JGenRendererValve"/>
> > </when>
> > </select>
> > </valves>
> > </pipeline>
>
> I think this is an excellent approach. Only problem I can see is what if
> there is no extension in the request. What if I want the result of a form
> submission to my FDF. I guess I could just put something like:
>
> action="$link.setAction(...).setPage(".FDF")"
>
> or something like that.
>
> > This gives users a ton of flexibility. We could implement some standard
> > selectors, for example ones that look at request parameters or objects
> > stored in RunData (however that looks when Jason finishes with it). The
> > base classes for selectors could be more intelligent, to allow globbed
> > or regexp matches.
>
> yes. yes. yes.
>
> > In Gareth's case, this would allow him a few ways to accomplish his
> > goal. He could simply leave out any target valve for certain extensions,
>
> No I don't like this, I would prefer to do all rendering in a
> target valve.
> This makes more sense in the design. I guess I just need to make sure that
> the relevant data in the the session.
>
> > Or (the one I prefer) he could implement a valve for
> > rendering using the FDF sdk, and use a selector to ensure that that
> > valve is run for the relevent targets.
> >
>
> Yes I prefer this but its difficult to see how it would work:
>
> ACTION:
>
> 1.Parse Form data
> 2.Put it in session?
>
> VALVE:
>
> 1. Get data from session
> 2. Render appropriate FDF File
>
> ????
>
> Not really sure about this approach. At the moment the result of
> the action
> is a byte[] (or String actually) which represents the FDF. I could store
> just this output in the session. Then print it to the output stream in the
> valve. Seems like a weird way to pass data around though.
>
> What to you think?
>
> Gareth
>
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>