Send-Ahead

[Jörg Walter <jw...@garni.ch>]

Hi!

This is a call for discussion. My current project involving big web forms 
(selects having tens or hundreds of entries, all fields explained by title="" 
attributes, ...) has a definite slowness experience. Using precaching, XSP 
execution time is 1-2 seconds, that's okay, but it takes 5 seconds after 
axkit declares delivery in the logs until the browser shows anything 
(everything at once, to be exact).
One big advantage of PHP, CGI, ... is the ability to send the http headers and 
some content ahead of the rest - even before the main grunt of processing 
started. It makes AxKit appear a lot slower, although it isn't.

My question: how can AxKit accomplish (something similar to) that?

My only idea so far would be to use a classic sandwich-template system and 
integrate AxKit through Apache::Filter. Which would mostly defy the reason to 
use AxKit. Send-Ahead is only useful for noncacheable resources.

Please tell me your ideas, they may be a bit academic, hard to implement, or 
not possible right now, still I'd like to hear them.

-- 
CU
   Joerg

Re: Send-Ahead

[Michael Chamberlain <mi...@btinternet.com>]

On Friday, October 3, 2003, at 08:56  am, Jörg Walter wrote:
>
> This is one possibility I have thought about, but it feels a lot like 
> ye
> olde sandwich templates would be resurrected for the last step. Feel 
> free to
> (verbally) prove me wrong, however.
>

Yeah, it would be a sandwich template. (would have to be if your using
dom/xslt for your middle bit). But it would be pipeline configurable,
so you could do different ones for different pages, and it could call
the pipeline multiple times if you wanted to use xslt in a less chunky
way.

Apache2 should also handle it nicely.


> One idea I had was to use a pure SAX pipeline. Online discussions have
> resulted in the idea to provide a get_sax method in addition to 
> get_dom and
> get_strref. get_sax would be a little bit different, as it wouldn't 
> really
> "get" anything, but pass the own SAX hander to theupstream SAX 
> handler, so
> that both modules can connect as a plain old SAX pipeline. That 
> pipeline is
> easily capable of send-ahead, since it processes one element at a 
> time. If
> we get a decent language module which exploits this behaviour (in 
> contrast
> to "collect all events, build a DOM, ..."), that would be a feasible 
> way to
> do it. I don't recall it off the top of my head, but aren't there SAX
> transformation modules that are used much like a stripped-down XSLT?
>
>

That's also possible. :)


Mike.

Re: Send-Ahead

[J�rg Walter <jw...@gatekeeper.garni.ch>]

On Thu, Oct 02, 2003 at 07:09:46PM +0100, Michael Chamberlain wrote:
> 
> On Thursday, October 2, 2003, at 03:32  pm, Robin Berjon wrote:
> 
> >J?rg Walter wrote:
> >>My question: how can AxKit accomplish (something similar to) that?
> >>My only idea so far would be to use a classic sandwich-template 
> >>system and integrate AxKit through Apache::Filter. Which would mostly 
> >>defy the reason to use AxKit. Send-Ahead is only useful for 
> >>noncacheable resources.
> >>Please tell me your ideas, they may be a bit academic, hard to 
> >>implement, or not possible right now, still I'd like to hear them.
> >
> >Well the problem here would be that you can't know what headers to 
> >send before the last transformation (since it might affect at least 
> >the mime type), and since the last transform returns its content as a 
> >whole (as opposed to progressively outputting it) I'm not sure how 
> >much you'd gain by sending the headers a little bit earlier than we do 
> >now. Or have I misunderstood the issue?
> >
> 
> Thinking about it in relation to my pipeline stuff, it might be 
> possible if we insist it's the last
> thing in the transformation list. Since it gets called in reverse it's 
> the first thing called.

This is one possibility I have thought about, but it feels a lot like ye
olde sandwich templates would be resurrected for the last step. Feel free to
(verbally) prove me wrong, however.

One idea I had was to use a pure SAX pipeline. Online discussions have
resulted in the idea to provide a get_sax method in addition to get_dom and
get_strref. get_sax would be a little bit different, as it wouldn't really
"get" anything, but pass the own SAX hander to theupstream SAX handler, so
that both modules can connect as a plain old SAX pipeline. That pipeline is
easily capable of send-ahead, since it processes one element at a time. If
we get a decent language module which exploits this behaviour (in contrast
to "collect all events, build a DOM, ..."), that would be a feasible way to
do it. I don't recall it off the top of my head, but aren't there SAX
transformation modules that are used much like a stripped-down XSLT?

-- 
CU
	Joerg

PGP Public Key at http://ich.bin.kein.hoschi.de/~trouble/public_key.asc
PGP Key fingerprint = D34F 57C4 99D8 8F16 E16E  7779 CDDC 41A4 4C48 6F94

Re: Send-Ahead

[Jörg Walter <jw...@garni.ch>]

On Sunday, 05. October 2003 21:01, Robin Berjon wrote:

> I was about to say that send-ahead should be the default since it makes a
> site seem more responsive but there's the issue of last steps not setting
> the mime-type themselves but instead relying on a previous step. This is
> never the case in XSLT (or is it?) but could be in other situations.
>
> So the question is, are those "other situations" worth taking into
> consideration (I can't find a compelling use case) and even if they are,
> could we still make send-ahead the default, and document that you have to
> turn it off (or provide a dummy last step) if you want an earlier bit of
> the pipeline to set the mime-type?

Yes they are, very much so - think about redirects. You can't do a redirect 
from XSP when your final provider already sent the 200 OK + headers. One 
thing I can imagine is a processor flag, like "sub deny_sendahead", which 
would return true if there is a reason not to sendahead. An XSP page would 
return true if the XSP contains a <web:redirect/> tag, for example. (The XSP 
source is available and could be parsed before the pipeline is run, so this 
is possible.) The last provider would then do "if 
(!$self->upstream->deny_sendahead()) { ... }"

OTOH, Content types set by earlier language modules are something we shouldn't 
support, IMHO. AxKit is an XML transformation framework, not a generic data 
transformation framework, so having anything but text/xml in any step but the 
last looks pointless to me (and asking for trouble - think encodings, UTF-8, 
binary data, ...).

-- 
CU
  Joerg

PGP Public Key at http://ich.bin.kein.hoschi.de/~trouble/public_key.asc
PGP Key fingerprint = D34F 57C4 99D8 8F16 E16E  7779 CDDC 41A4 4C48 6F94

Re: Send-Ahead

[Michael Chamberlain <mi...@btinternet.com>]

>>
>
> I was about to say that send-ahead should be the default since it 
> makes a site seem more responsive but there's the issue of last steps 
> not setting the mime-type themselves but instead relying on a previous 
> step. This is never the case in XSLT (or is it?) but could be in other 
> situations.
>

The last xslt stylesheet is asked for it's output_encoding() when you 
transform to a string.
Since it might be html that comes out, by definition it must be the 
last xml based transformation step.

But you could have say the pdf transform following something like that.

For things like xslt and pdf, we can ask in advance the pipeline what 
it's output is going to be,
for something like xsp/xpantscript as the last thing, it's probably 
less definable. In which case
the correct thing to do would be disable send ahead in those situations.

> So the question is, are those "other situations" worth taking into 
> consideration (I can't find a compelling use case) and even if they 
> are, could we still make send-ahead the default, and document that you 
> have to turn it off (or provide a dummy last step) if you want an 
> earlier bit of the pipeline to set the mime-type?

we can make it a pipeline option or a notes setting it checks. Send 
ahead is really going to be a sax
thing, we can send some headers for dom stuff, but not many.

PS: Considering this would have to be on top of the pipeline stuff, do 
we want to reconsider not commiting
it into axkit1? or is this still aimed at axkit2.

PPS: If it's axkit2 can someone who knows how do some magic branch 
stuff on the cvs tree so I can
commit stuff there. (or tell me what I need to do).

Mike.

Re: Send-Ahead

[Robin Berjon <ro...@knowscape.com>]

Michael Chamberlain wrote:
> On Thursday, October 2, 2003, at 03:32  pm, Robin Berjon wrote:
>> Jörg Walter wrote:
>>
>>> My question: how can AxKit accomplish (something similar to) that?
>>> My only idea so far would be to use a classic sandwich-template 
>>> system and integrate AxKit through Apache::Filter. Which would mostly 
>>> defy the reason to use AxKit. Send-Ahead is only useful for 
>>> noncacheable resources.
>>> Please tell me your ideas, they may be a bit academic, hard to 
>>> implement, or not possible right now, still I'd like to hear them.
>>
>> Well the problem here would be that you can't know what headers to 
>> send before the last transformation (since it might affect at least 
>> the mime type), and since the last transform returns its content as a 
>> whole (as opposed to progressively outputting it) I'm not sure how 
>> much you'd gain by sending the headers a little bit earlier than we do 
>> now. Or have I misunderstood the issue?
> 
> Thinking about it in relation to my pipeline stuff, it might be possible 
> if we insist it's the last
> thing in the transformation list. Since it gets called in reverse it's 
> the first thing called.

Ah but yes, this makes for a most excellent first step :)

> We just have to ensure the last thing in the pipeline is a 
> Pipeline::Sender or such, if one exists
> do nothing, if not add an axkit standard one. Then you just make your 
> send ahead thing a
> inherit from pipeline::sender....

I was about to say that send-ahead should be the default since it makes a site 
seem more responsive but there's the issue of last steps not setting the 
mime-type themselves but instead relying on a previous step. This is never the 
case in XSLT (or is it?) but could be in other situations.

So the question is, are those "other situations" worth taking into consideration 
(I can't find a compelling use case) and even if they are, could we still make 
send-ahead the default, and document that you have to turn it off (or provide a 
dummy last step) if you want an earlier bit of the pipeline to set the mime-type?

Re: Send-Ahead

[Michael Chamberlain <mi...@btinternet.com>]

On Thursday, October 2, 2003, at 03:32  pm, Robin Berjon wrote:

> Jörg Walter wrote:
>> My question: how can AxKit accomplish (something similar to) that?
>> My only idea so far would be to use a classic sandwich-template 
>> system and integrate AxKit through Apache::Filter. Which would mostly 
>> defy the reason to use AxKit. Send-Ahead is only useful for 
>> noncacheable resources.
>> Please tell me your ideas, they may be a bit academic, hard to 
>> implement, or not possible right now, still I'd like to hear them.
>
> Well the problem here would be that you can't know what headers to 
> send before the last transformation (since it might affect at least 
> the mime type), and since the last transform returns its content as a 
> whole (as opposed to progressively outputting it) I'm not sure how 
> much you'd gain by sending the headers a little bit earlier than we do 
> now. Or have I misunderstood the issue?
>

Thinking about it in relation to my pipeline stuff, it might be 
possible if we insist it's the last
thing in the transformation list. Since it gets called in reverse it's 
the first thing called.

We just have to ensure the last thing in the pipeline is a 
Pipeline::Sender or such, if one exists
do nothing, if not add an axkit standard one. Then you just make your 
send ahead thing a
inherit from pipeline::sender....

Mike.

Re: Send-Ahead

[Robin Berjon <ro...@knowscape.com>]

Jörg Walter wrote:
> My question: how can AxKit accomplish (something similar to) that?
> 
> My only idea so far would be to use a classic sandwich-template system and 
> integrate AxKit through Apache::Filter. Which would mostly defy the reason to 
> use AxKit. Send-Ahead is only useful for noncacheable resources.
> 
> Please tell me your ideas, they may be a bit academic, hard to implement, or 
> not possible right now, still I'd like to hear them.

Well the problem here would be that you can't know what headers to send before 
the last transformation (since it might affect at least the mime type), and 
since the last transform returns its content as a whole (as opposed to 
progressively outputting it) I'm not sure how much you'd gain by sending the 
headers a little bit earlier than we do now. Or have I misunderstood the issue?