Changing the Pipeline Model

[Matt Sergeant <ma...@sergeant.org>]

> (A push pipeline, each stage is run independantly, the output pushed
> to the next handler).
>
> A pull pipeline looks just like a function call stack, starting at the
> outside... which means each stage ask the stage before it for the
> representation it wants before getting it, which allows for lots of
> self optimisation.
>
> send( get_binary ( get_text ( get_dom( get_dom( get_dom( ) ) ) ) ) )
>
> (in the code this is the $self->upstream()->get_dom() stuff).
>
> The (dis)advantage is that the pipeline gets created first, before 
> things
> like the cache get created, which allow for the cache to interact with 
> the
> pipline (and thus XSP). The disdvantage is is that it's a little 
> slower than
> the current axkit due to this overhead.
>
> It however also means that we can make the cache just another pipeline 
> module.
> So incremental caching is available if you want it out of the box.
>
> AxKit.pm is a lot smaller.
>
> Chained XSP works.
>
> Chained sax handlers work the way you would expect them to, they're 
> just a style
> using the Apache::AxKit::Pipeline::SAX processor
>
> Error stylesheets have been fixed, there is an Error Provider now 
> which you need
> to set up as you server 500 handler.

Before I chime in on Kip's "Wake-Up" thread, which I will try and do 
tomorrow, I want to address this issue, and start off a new thread on 
it, so the WakeUp call doesn't get bogged down in discussing this side 
issue.

We need to come to a decision on this - whether to fundamentally change 
our pipeline model in AxKit-2.0.

As I see it the arguments for/against are something like this:

For:
  - The pull model cleans up some code
  - The pull model already supports some things we've wanted for a 
while: e.g. Chained XSP and more flexible cache
  - The pull model means some cleaner code

Against:
  - It's a big change. This means a change for developers and users 
alike to get used to.
    - (Meta For argument) - 2.0 is as good a time as any for big changes
  - Some of the things the pull model does should be possible with a bit 
more work using the push model, without breaking anything else.
  - There's no easy way to change the pipeline mid-stream with a pull 
model

I don't exactly know the right answer to this, so if you have input 
into this send it now. I think what really has to happen is we have to 
download this new code and play with it, and see what we think of it 
from an individual point of view, and then come back here (and not IRC) 
and discuss it openly.

Matt.

Re: Changing the Pipeline Model

[Mike Chamberlain <mi...@btinternet.com>]

>
>  - There's no easy way to change the pipeline mid-stream with a pull 
> model
>

There is code in there now to allow append to the front (from current 
position),
and append to the end. The problem comes if you want to add stuff in 
the middle :)
But there is ways to do that if you know in advance where in the middle 
you want
to add stuff. :)

> I don't exactly know the right answer to this, so if you have input 
> into this send it now. I think what really has to happen is we have to 
> download this new code and play with it, and see what we think of it 
> from an individual point of view, and then come back here (and not 
> IRC) and discuss it openly.
>
> Matt.
>

That'd be cool.

Mike.

Re: Changing the Pipeline Model

[Tom Schindl <to...@gmx.at>]

Matt Sergeant wrote:
>> (A push pipeline, each stage is run independantly, the output pushed
>> to the next handler).
>>
>> A pull pipeline looks just like a function call stack, starting at the
>> outside... which means each stage ask the stage before it for the
>> representation it wants before getting it, which allows for lots of
>> self optimisation.
>>
>> send( get_binary ( get_text ( get_dom( get_dom( get_dom( ) ) ) ) ) )
>>
>> (in the code this is the $self->upstream()->get_dom() stuff).
>>
>> The (dis)advantage is that the pipeline gets created first, before things
>> like the cache get created, which allow for the cache to interact with 
>> the
>> pipline (and thus XSP). The disdvantage is is that it's a little 
>> slower than
>> the current axkit due to this overhead.
>>
>> It however also means that we can make the cache just another pipeline 
>> module.
>> So incremental caching is available if you want it out of the box.
>>
>> AxKit.pm is a lot smaller.
>>
>> Chained XSP works.
>>
>> Chained sax handlers work the way you would expect them to, they're 
>> just a style
>> using the Apache::AxKit::Pipeline::SAX processor
>>
>> Error stylesheets have been fixed, there is an Error Provider now 
>> which you need
>> to set up as you server 500 handler.
> 
> 
> Before I chime in on Kip's "Wake-Up" thread, which I will try and do 
> tomorrow, I want to address this issue, and start off a new thread on 
> it, so the WakeUp call doesn't get bogged down in discussing this side 
> issue.
> 
> We need to come to a decision on this - whether to fundamentally change 
> our pipeline model in AxKit-2.0.
> 
> As I see it the arguments for/against are something like this:
> 
> For:
>  - The pull model cleans up some code
>  - The pull model already supports some things we've wanted for a while: 
> e.g. Chained XSP and more flexible cache

I'm working on chained XSP by using my new XML::LibXSLT::Enhanced(a new 
module of mine) which allows you to add perl functions inside your XSL. 
As a proof of concept I reimplemented your ESQL-Taglib. With this 
approach you can easily chain XSP because it's nothing more than an 
XSL(T)-Pipeline.

At the moment the following things are supported:
* defining your own variables (to overcome: you only set a variable once
   in xsl)
* logging via log4perl (that's really fantastic to debug XSLs)
* esql (not competely ready yet)

It also comes with a commandline interface named xslt-proc-enhanced 
which allows you to register TagLib's out side of the AxKit-Context and 
test your xsls like one is used to with xslt-proc.

The module is not ready for release on CPAN still if somebody is 
interessted in it I can publish the code. At the moment one can also use 
http://search.cpan.org/~tomson/Apache-AxKit-Language-LibXSLTEnhanced-0.02/ 
  this was my first try on adding something like TagLib's to XML::LibXSLT.

Please note when writing TagLib's using this module they will not be 
compatible with XML::LibXSLT::Enhanced-TagLibs.


>  - The pull model means some cleaner code
> 
> Against:
>  - It's a big change. This means a change for developers and users alike 
> to get used to.
>    - (Meta For argument) - 2.0 is as good a time as any for big changes
>  - Some of the things the pull model does should be possible with a bit 
> more work using the push model, without breaking anything else.
>  - There's no easy way to change the pipeline mid-stream with a pull model
> 
> I don't exactly know the right answer to this, so if you have input into 
> this send it now. I think what really has to happen is we have to 
> download this new code and play with it, and see what we think of it 
> from an individual point of view, and then come back here (and not IRC) 
> and discuss it openly.
> 
> Matt.
> 
> 

Re: Changing the Pipeline Model

[Mike Chamberlain <mi...@btinternet.com>]

 --- Matt Sergeant <ma...@sergeant.org> wrote: > On 22
Apr 2004, at 21:18, Mike Chamberlain wrote:
> 
> > Well the config reader would obviously have to
> support the option in 
> > the same way it supports
> > other Ax* directives...
> 
> What I meant was, would a different pipeline model
> require a completely 
> different looking config file, and thus be tied to a
> configreader.
> 

Ah, I see what you mean.

OK, at the moment I've only required the one extra
directive, but then I've not tried to do anything
clever with it.

I'm guessing there is no easy way which would allow
modules to define their own configuration options, and
some how extend the config reader when they're used.

That would be really cool, but require quite a bit of
thought into that api.

Mike.

Re: Changing the Pipeline Model

[Matt Sergeant <ma...@sergeant.org>]

On 22 Apr 2004, at 21:18, Mike Chamberlain wrote:

> Well the config reader would obviously have to support the option in 
> the same way it supports
> other Ax* directives...

What I meant was, would a different pipeline model require a completely 
different looking config file, and thus be tied to a configreader.

Matt.

Re: Changing the Pipeline Model

[Mike Chamberlain <mi...@btinternet.com>]

On 22 Apr 2004, at 21:04, Matt Sergeant wrote:

> On 22 Apr 2004, at 17:08, Kip Hampton wrote:
>
>> Matt Sergeant wrote:
>>
>>> We need to come to a decision on this - whether to fundamentally 
>>> change our pipeline model in AxKit-2.0.
>>
>> I think the only sane answer to "do we do implement a pull or a push 
>> model for the core pipelines?" is "Yes!". What I mean is: why don't 
>> we make the pipeline just YA swappable component rather than 
>> hard-coding one model or the other?
>>
>> Thoughta?
>
> My thoughta is this might be needless complexity (?). Though I really 
> do need to sit down with the code and play some. Would making the 
> pipeline swappable make it incompatible with the ConfigReader (or 
> rather, make it tied to the ConfigReader)?
>
>

Well the config reader would obviously have to support the option in 
the same way it supports
other Ax* directives... But I don't think it adds needless complexity. 
Theoretically it should
make axkit more powerful, by allowing people to more interesting stuff 
than just a straight pipeline.

After all, almost everything else is plugable, including the config 
reader, so why not the pipeline?

Mike.

Re: Changing the Pipeline Model

[Matt Sergeant <ma...@sergeant.org>]

On 22 Apr 2004, at 17:08, Kip Hampton wrote:

> Matt Sergeant wrote:
>
>> We need to come to a decision on this - whether to fundamentally 
>> change our pipeline model in AxKit-2.0.
>
> I think the only sane answer to "do we do implement a pull or a push 
> model for the core pipelines?" is "Yes!". What I mean is: why don't we 
> make the pipeline just YA swappable component rather than hard-coding 
> one model or the other?
>
> Thoughta?

My thoughta is this might be needless complexity (?). Though I really 
do need to sit down with the code and play some. Would making the 
pipeline swappable make it incompatible with the ConfigReader (or 
rather, make it tied to the ConfigReader)?

Matt.

Re: Changing the Pipeline Model

[Mike Chamberlain <mi...@btinternet.com>]

 --- Kip Hampton <kh...@totalcinema.com> wrote: >
Matt Sergeant wrote:
> 
> > We need to come to a decision on this - whether to
> fundamentally change 
> > our pipeline model in AxKit-2.0.
> 
> I think the only sane answer to "do we do implement
> a pull or a push 
> model for the core pipelines?" is "Yes!". What I
> mean is: why don't we 
> make the pipeline just YA swappable component rather
> than hard-coding 
> one model or the other?
> 
> Thoughta?
> 

I've actually done some work towards that goal on the
pipeline branch, but not yet commited it. Shouldn't be
too much more to do on it. 

Ideally then the way the pipeline works (and the
features available) would be swappable. 

So definite +1 from me with regards making the
pipeline controller just another configurable part.

Mike.

Re: Changing the Pipeline Model

[Kip Hampton <kh...@totalcinema.com>]

Matt Sergeant wrote:

> We need to come to a decision on this - whether to fundamentally change 
> our pipeline model in AxKit-2.0.

I think the only sane answer to "do we do implement a pull or a push 
model for the core pipelines?" is "Yes!". What I mean is: why don't we 
make the pipeline just YA swappable component rather than hard-coding 
one model or the other?

Thoughta?

-kip