xsp Serverpages generator: Convoluted functionality?

[Leon Widdershoven <qa...@dds.nl>]

Hi,

<abstract>
   I have a question about the design of the ServerPages generator, which
   transforms xsp pages into (at the end) a SAX stream.
   I think the generator wraps to distinct functions into one function,
   and that it could be usefull to split it into a File-like Reader into
   a SAX stream and a separate Transformer which transforms SAX via a
   compilation to another SAX stream.
</abstract>

<pre>
In this e-mail I address a problem which is bugging me almost from the 
start. I have looked at the mailing list but could not find such a 
thread. Whether this is because my idea is stupid or not, I do not know. 
I mean no critique, but i *am* curious.
</pre>

I have noticed that the XSP generator has multiple functions wrapped 
into one:
1. Reading the xsp file (into a DOM tree probably)
2. Transforming this tree into Java
3. Compiling the resulting java file
4. executing this file

After 4) the results is passed further in the pipeline.

I have also noticed that this design results in quite some unnecessary 
trouble for some users, including myself. I try to use the session
authentication data in my xsp pages. But that, naturally, fails since
the session transformer works on the stream that results from the 
serverpages generator. In other words, after the xsp has been compiled.

This means that values from the authentication context cannot be used; 
in effect it means that all logic sheets which do not bind as well to
xsp as esql cannot be used to control the java control flow.

I think this can be done better. It I look at the design concept of the 
sitemap it has generators to load data and transform it into a SAX 
stream, which then is processed by a transformer.

The transformer takes a SAX stream, does it's thing, and returns a SAX 
stream, leaving intact all those tags it can not handle.

At the end the serializer transforms it into the output file which may 
not be a DOM stream.

It I try to map the ServerPages generator into that framework I notice 
it does two separate things:

1. It generates a SAX tree from an xsp file

2. It transforms a SAX tree into a java file, executes that file, and
    returns a SAX tree

Though the naming is quite suggestive, I think the point is clear. I 
think the ServerPages generator should actually be split into two parts. 
That would enable other transformers to handle their tags, after which 
the Java control flow can take that data into account. Such a construct 
would allow non-xsp tags to integrate seamlessly with Java (xsp) programs.

Such a separation of functionality, however, is just so natural that I 
really can't imagine that no-one else thought of that. So I'm actually 
quite curious why such a split has not been done.

I'm also quite curious how the actual implementation of the generator 
works. I have looked at the source code of some xsp tag libraries to get 
other things to work, but I'm a bit scared of the serverpages generator 
- it just has to be complicated. I'm going to look at it anyway in the 
hope that internally the reading of the XML stream and the 
transformation into java is nicely separated, so that I can re-write it 
into two components. If the separation is clean, I think I can do that 
faster than writing my xsl's to handle session-transformed data.

I don't have high hopes, however, so I would really like some comments 
from the people who actually wrote the generator.

Regards,
Leon Widdershoven

Re: xsp Serverpages generator: Convoluted functionality?

[Upayavira <uv...@upaya.co.uk>]

Leon,

Thank you for your detailed consideration of the ServerPages generator.

As I understand it, the ServerPages generator is primarily used with 
XSP. These days, XSP isn't a primary and recommended tool within Cocoon. 
It is primarily there to support existing users. Alternative methods are 
available that should enable you to achieve everything you can do in 
XSP, such as FlowScript and JXTemplate.

Therefore, whilst I do sense some merit in what you write, I do not 
expect anyone here to be interested in modifying or rewriting the 
ServerPages generator. It works, and that, for us, is good enough.

If you wish to carry on using XSP, that's fair enough - otherwise I'd 
encourage you to try Flowscript and its associated methods (including 
the CompilingClassLoader that allows flowscripts to use Java code that 
is compiled on the fly, without restarting your servlet container). If 
you start using these technologies, then you'd be more than welcome to 
join us in the development effotrt that is going on around improving 
them, and in terms of learning how best to use them.

Hope this helps, and is clear enough.

With best wishes,

Upayavira

Leon Widdershoven wrote:

> Hi,
>
> <abstract>
>   I have a question about the design of the ServerPages generator, which
>   transforms xsp pages into (at the end) a SAX stream.
>   I think the generator wraps to distinct functions into one function,
>   and that it could be usefull to split it into a File-like Reader into
>   a SAX stream and a separate Transformer which transforms SAX via a
>   compilation to another SAX stream.
> </abstract>
>
> <pre>
> In this e-mail I address a problem which is bugging me almost from the 
> start. I have looked at the mailing list but could not find such a 
> thread. Whether this is because my idea is stupid or not, I do not 
> know. I mean no critique, but i *am* curious.
> </pre>
>
> I have noticed that the XSP generator has multiple functions wrapped 
> into one:
> 1. Reading the xsp file (into a DOM tree probably)
> 2. Transforming this tree into Java
> 3. Compiling the resulting java file
> 4. executing this file
>
> After 4) the results is passed further in the pipeline.
>
> I have also noticed that this design results in quite some unnecessary 
> trouble for some users, including myself. I try to use the session
> authentication data in my xsp pages. But that, naturally, fails since
> the session transformer works on the stream that results from the 
> serverpages generator. In other words, after the xsp has been compiled.
>
> This means that values from the authentication context cannot be used; 
> in effect it means that all logic sheets which do not bind as well to
> xsp as esql cannot be used to control the java control flow.
>
> I think this can be done better. It I look at the design concept of 
> the sitemap it has generators to load data and transform it into a SAX 
> stream, which then is processed by a transformer.
>
> The transformer takes a SAX stream, does it's thing, and returns a SAX 
> stream, leaving intact all those tags it can not handle.
>
> At the end the serializer transforms it into the output file which may 
> not be a DOM stream.
>
> It I try to map the ServerPages generator into that framework I notice 
> it does two separate things:
>
> 1. It generates a SAX tree from an xsp file
>
> 2. It transforms a SAX tree into a java file, executes that file, and
>    returns a SAX tree
>
> Though the naming is quite suggestive, I think the point is clear. I 
> think the ServerPages generator should actually be split into two 
> parts. That would enable other transformers to handle their tags, 
> after which the Java control flow can take that data into account. 
> Such a construct would allow non-xsp tags to integrate seamlessly with 
> Java (xsp) programs.
>
> Such a separation of functionality, however, is just so natural that I 
> really can't imagine that no-one else thought of that. So I'm actually 
> quite curious why such a split has not been done.
>
> I'm also quite curious how the actual implementation of the generator 
> works. I have looked at the source code of some xsp tag libraries to 
> get other things to work, but I'm a bit scared of the serverpages 
> generator - it just has to be complicated. I'm going to look at it 
> anyway in the hope that internally the reading of the XML stream and 
> the transformation into java is nicely separated, so that I can 
> re-write it into two components. If the separation is clean, I think I 
> can do that faster than writing my xsl's to handle session-transformed 
> data.
>
> I don't have high hopes, however, so I would really like some comments 
> from the people who actually wrote the generator.
>
> Regards,
> Leon Widdershoven
>
>
>
>
>
>
>

Re: xsp Serverpages generator: Convoluted functionality?

[Vadim Gritsenko <va...@reverycodes.com>]

Leon Widdershoven wrote:

> Hi,


I read all this email but it leaves me with feeling that you have not 
got yet how XSP are working... Here goes my attempt at explaining how 
they work:

ServerPagesGenerator is responsible for:
  (1) Loading Generator created out of ServerPage
  (2) Invoking Generator created out of ServerPage

Step (1) includes:
  * Choosing markup laguage (usually: XSP)
  * Choosing programming language (Java, Javascript, Python)
  * Resolving serverpages Source
  * Invoking ProgramGenerator to create Generator out of Source in given 
markup, programming language pair (3)

Step (2) is just one-liner, invoke generate() method on obtained 
generator, plus some exception handling, etc.


Now, as you can see, actual creation of the Generator happens outside of 
the ServerPagesGenerator, and done by ProgramGenerator. 
ProgramGenerator's responsibilities include:
  * Selecting a Generator instance, if it is loaded already
  * Loading a Generator instance, if it is compiled already
  * Generate a source code (and compile it) out of the server page.

Usually, last step happens only on first access to the server page. 
Second step (usually) happens on first access to the server page after 
server restart. First step happens on each access to the serverpage.

Now, as you can see, programming language source code is generated out 
of XSP source code only once, on first access to the server page (also, 
it is done each time XSP source is modified). On all other accesses to 
the server page, already loaded Generator instance is used, and no 
compilation take place. If you think about it, that's the ONLY possible 
way to implement server pages, otherwise the only thing server will be 
doing is invoking compiler.

When source code is generated out of server page in markup language, it 
is possible to insert arbitrary XSLT transformations on this code. Such 
XSLT transformations are called Logicsheets, and they are replacing tags 
in the logicsheet's namespace with the logic in programming language. 
Thus, for example,
     <xsp-request:get-parameter name="test"/>
becomes:
     request.getParameter("test")
after XSP source is passed through the XSP Request Logicsheet.

Now, suppose that you found a way to use SessionTransformer. And suppose 
that you had asked it to get some information from the current user's 
session. This means, that whatever information is extracted by the 
SessionTransformer, will go into the server page's source code in the 
programming language of choice. And this information will be "baked 
into" the code of the resulting Generator. Next, what will happen when 
another user accesses this same server page? There are two options:
  (a) Another user will get data of the first user.
  (b) You will have to make server pages generator to discard compiled 
instance of the server page and compile it again.

I think (a) is not what you want. And also (b) is possible, it would 
NEVER scale beside a cute little demo (see 
http://localhost:8888/samples/sources/all-dynamic demo). And that's 
where Logicsheets are coming in. What you need is a logicsheet, which 
will inject logic into the programming code of the server page, which 
will invoke necessary Java code to obtain information from the session - 
see example above about xsp-request.


And last comment about server pages technology in Cocoon. It is mature, 
and no deep changes are planned for it. Cocoon community concentrating 
on other, more promising (but - sometimes - unstable yet) technologies, 
and no significant time will be spent on development of server pages 
technology. Core of the server pages technology is very solid, but some 
of the logicsheets are still in need of polishing. Now, you can either 
join and work on developing newer technologies, such as jxtemplates, or, 
if you are not in position to do this, your contributions (bugfixes) to 
existing XSP logicsheets are more than welcome.

Thanks,
Vadim