Re: [Design] JXTG 2.0 (Just say you're sorry!)

[Miles Elam <mi...@pcextremist.com>]

First of all, I would like to apologize for my inflammatory tone.  My 
ire was directed toward the proposal, not the people proposing it.  
That said, I went over the top.  Too many arguments with young Earth 
Creationists lately and it spilled over into my writing on this list.  
To Leszek in particular, I am sorry for the reckless criticism.  To 
Stefano, I'm sorry for putting you in the position of agreeing with 
someone who did not conduct himself well.

--------

But I have seen taglibs go terribly wrong in real-world situations far 
more often than they've worked out.  I'm a firm believer in defining 
the interface first through use cases and only afterward discussing the 
implementation/design.  Granted, my primary experience with taglibs is 
JSP's flavor.  That may have colored my view of the technology unfairly 
in general.


On Dec 2, 2004, at 12:54 PM, Daniel Fagerstrom wrote:

> Let me re-iterate: there have for a long time been a concesus at the 
> list among those who have cared enough to discuss it that JXTG is a 
> well working way of creating views, but that the implementation is 
> very hard to maintain.

Granted, but citing that JXTG is a monolithic headache doesn't mean 
that the deconstruction must be total.  Modularization can happen in 
degrees.  For example, just by making a common interface between lookup 
libraries (JXPath, JEXL, etc.) would remove HUGE amounts of code from 
JXTG.  The switch back and forth between object lookup mechanisms is 
probably the biggest reason for JXTG's complexity.

In addition, I've had virtual sitemap component possibilities floating 
through my head lately.  Having a generator like:

<map:generator name="example">
   <map:absolutize param="templatename"/>

   <map:generate type="file" src="{templatename}.xml"/>
   <map:transform type="stx" src="macros.stx"/>
   <map:transform type="datainject"/><!-- Yes, I know this doesn't exist 
now -->
</map:generator>

This strikes me as doing very much what is expected from taglibs.  The 
advantages I see are that (a) it's easier for a site administrator to 
see at a glance what goes on and (b) keeps things sane from an 
implementation standpoint.  What do I mean by 'b'?  From my experiences 
with JSP taglibs, any taglib must either be absolutely self-contained 
or fundamentally aware of all other taglibs in use.  Mixing them in the 
same document blows some assumptions about use out the window in 
certain cases.  Debugging and reworking them ends up being 
fundamentally similar to writing raw code.

Other than for the purposes of mixing logic from different sources 
simultaneously and praying they cause no mutual compatibility/state 
problems, I don't see the advantage over transformers.  In addition, if 
attributes were trigger items (attrlibs?), what determines precedence 
when there are two different attrlibs at work?

<foo data:bar="action" macro:bar="action"/>

XML parsers are under no obligation to enforce the document order of 
attributes.  What happens either way?  Does data injection happen first 
or macro expansion?  What if the element is dynamically renamed or 
conditional?  On the other hand, if you predetermine precedence, what 
advantage is there over transformers?  Remember, tag apis are no less 
code than transformer code.  Programming is programming.  Only the API 
varies.

> There has also been an agreement about that ESQL is the only reason 
> (besides back compability) to care about XSP, you said so youself a 
> week ago or so.

But is ESQL is the best choice?  Data input becomes a big deal when you 
use ESQL for output and then a stream generator combined with 
transformation combined with sqltransformer combined with...

I'm a big proponent of "make the better decision easier than the worse 
decision."  Quick and dirty, ESQL works.  For limited obscure cases, 
ESQL is great.  But the better decision would be to use object mapping 
(Hibernate, JDO, et al.) and CForms in Flow.  Tag libraries seem to me 
to promote an excessive shift of logic to the templating language 
instead of in the backend data processing architecture.  Not forcing 
but encouraging people to move as much logic as possible to Flow seems 
like it would allow more flexibility while remaining more maintainable 
and clear.  "Clear" not in the sense that the individual template and 
app is understandable but that the hundredth template and app is 
understandable.

> For those of us who use CForms it is very convenient to be able to use 
> the template tags together with JXTG constructs.
>
> So we need a template generator with three sets of tags "jx:", "esql:" 
> and "ft:" thats it.
>
> We also discused how to use a separate conversion layer to remove the 
> need for formating constructs from the template layer.

I'm not following.  If you are using CForms (and thus Flow), why would 
you make template calls to a database?  Why not do the persistence 
logic in Flow right next to where you are sending and receiving form 
data?  From a logical point of view, I would think that these would be 
better served next to one another.

Regarding an earlier comment that not everyone knows Java but may know 
SQL, ESQL is indeed a programming language.  Just because it doesn't 
use '{' and '}' characters doesn't alleviate that fact.  A simple 
example of iterating a result set in Java or JavaScript would be as 
useful to a newcomer as the equivalent ESQL sample.  IMHO of course.

Redirecting via sendPage() on error also strikes me as simpler to grok 
and much more flexible than the error handling qualities of ESQL.  
Note: I'm not trying to knock ESQL.  I'm just saying that I think that 
most of the time, I don't think it's the best solution.

>                            --- o0o ---
>
> Given these general requirements that have been discussed again and 
> again at the list and also some more technical, performance driven 
> requirments, I steped forward and proposed a possible way of 
> implementing it. This design is based on a rather far going SoC in the 
> interest of keeping it maintainable. We have also identified a number 
> of templateing components that are needed in other places in Cocoon. 
> E.g. expression language and object model and therefore are worth 
> implementing as separate components. I also proposed implementing the 
> three sets of tags discussed above as taglibs instead of making their 
> interpretation being part of a huge monolitic SAX event handler as in 
> JXTG. Implementing it that way it is must easier to write test code 
> for the tags and generally easier to understand what is going on.

Individual tags, yes.  What about the interaction of several tag 
libraries?

>                            --- o0o ---
>
> Given this background, I was quite suprised when Miles Elam whent 
> ballistic about that I mentioned the word taglib and draw conclsions 
> about my intensions that are far and in several cases oposite from 
> what I feel and have written anything about.
>
> Anyway, if you have better suggestions about how to solve the above 
> requirements I'm all ears.

I was not justified in going ballistic.  I should have used a more even 
tone.  This is not a crusade.  It is a technical discussion.  I 
temporarily lost sight of that.  I hope to be more collaborative from 
here on.

>                            --- o0o ---
>
> Now over to commenting what you have written.
>
>> So, my other side thinks that having a scripted controller invoquing 
>> different templated views is a better solution.
>> In this case, do we need taglibs at all? no we don't. 
>> <esql:query>select * from blah</esql:query> sounds easy enough, but 
>> in real life it's more something like
>>      <esql:connection>
>>       <esql:pool>gump</esql:pool>
>>        <esql:execute-query>
>>          <esql:query>
>>            SELECT name,description FROM projects ORDER BY name
>>          </esql:query>
>>          <esql:results>
>>            <esql:row-results>
>>             <li>
>>              <a><xsp:attribute 
>> name="href">projects?name=<esql:get-string 
>> column="name"/></xsp:attribute><esql:get-string column="name"/></a> - 
>> <esql:get-string column="description"/>
>>             </li>
>>            </esql:row-results>
>>          </esql:results>
>>        </esql:execute-query>
>>       </esql:connection>
>> and *THIS IS THE SIMPLES QUERY I CAN THINK OF*!!!

Also note that the example doesn't have strong error handling 
capabilities either.

> In the general case I would rather write just the query with some 
> surrounding tags like in the SQLTransformer, get a simple standardized 
> XML serialization of the row set and then transform it to HTML in 
> XSLT. The only difference compared to the SQLTransformer would be that 
> I can combine it with JXTG constructions and insert query params in a 
> convinient way.
>
> If I would like to do everything in one step as you suggest above it 
> might look more like:
>
>    <esql:connection>
>     <esql:pool>gump</esql:pool>
>      <esql:execute-query>
>        <esql:query>
>          SELECT name,description FROM projects WHERE 
> projectleader=${cocoon.request-param.id}ORDER BY name
>        </esql:query>
>        <esql:results>
>          <esql:row-results>
>           <li>
>            <a href="projects?name=${$row.name}"/>${$row.name}</a> - 
> ${$row.description}
>           </li>
>          </esql:row-results>
>        </esql:results>
>      </esql:execute-query>
>    </esql:connection>
>
> As we are using the same kind of expression template mechanisms as in 
> JXTG. We could probably make the syntax more efficient and take away 
> some of the tag layers if we feel like that.

What are the advantages over a series of transformers that handle their 
specific problem spaces?

>> What I want is something like this:
>>  - request comes
>>  - sitemap gets it
>>  - matcher matches
>>  - controller is executed and populates beans in the request context
>>  - pipeline is invoqued with access to the request context
>>  - response goes
>> Now, this can happen right now in flow and JXtemplate. If we don't 
>> need state management, this is just like having a better action model 
>> with XSP-like code recompilation.
>
> Sure, I agree with all that. Only question is: where do I put my SQL 
> queries in the above scenario?

Flow (the controller) during processing and populating beans in the 
request context.

> As said above my only purpose with introducing taglibs is to increase 
> maintainability and the ability to test things. Take a look at the 
> Cocoon code base. Tons of code that hide what it does beyond all this 
> SAX event handling code. If we could refactor away some of that 
> intermingling, we would IMO make a step forward.

Complete agreement with the periodic need to refactor.  I just question 
that taglibs are the solution to the code hiding and intermingling 
problems.  Referring to the ESQL samples above, I even more strongly 
believe this today than I did yesterday.  If relational database access 
is that hard/lengthy to write, I am inclined to question the technique 
rather than the technology used to implement that technique.

Another question: how do you effectively cache templates with tag 
library implementations like ESQL?  With transformers, they only have 
one problem to deal with at a time.

- Miles Elam

P.S.  Did I mention yet that I was sorry?

Re: [Design] JXTG 2.0 (Just say you're sorry!)

[Leszek Gawron <lg...@mobilebox.pl>]

Miles Elam wrote:
> First of all, I would like to apologize for my inflammatory tone.  My 
> ire was directed toward the proposal, not the people proposing it.  That 
> said, I went over the top.  Too many arguments with young Earth 
> Creationists lately and it spilled over into my writing on this list.  
> To Leszek in particular, I am sorry for the reckless criticism.  To 
> Stefano, I'm sorry for putting you in the position of agreeing with 
> someone who did not conduct himself well.
No hard feelings Mr. Bond :). I am not even half experienced as you all 
are and participating in cocoon development is a pure pleasure - even 
listening to criticism. I learn from it A LOT.

One thing only to clarify: I am to blame for the commit. But I am not
the creator of the base cocoon-block-template implementation. I just
cleaned up Jonas Ekstedt's work - the credit for the effort goes to him.

> But I have seen taglibs go terribly wrong in real-world situations far 
> more often than they've worked out.  I'm a firm believer in defining the 
> interface first through use cases and only afterward discussing the 
> implementation/design.  Granted, my primary experience with taglibs is 
> JSP's flavor.  That may have colored my view of the technology unfairly 
> in general.
As I have said in other post: The taglib word may be very dangerous for
whole effort here. I think about this as of the renderers of the model
with no side effects. As for other taglibs - we should not host them in
cocoon distribution to give users bad impression of the direction they
should follow.

ESQL should not be implemented in new template. It has too much side 
effect and is a powerful weapon in anti-template camp hands' :)

> Granted, but citing that JXTG is a monolithic headache doesn't mean that 
> the deconstruction must be total.  Modularization can happen in 
> degrees.  For example, just by making a common interface between lookup 
> libraries (JXPath, JEXL, etc.) would remove HUGE amounts of code from 
> JXTG.  The switch back and forth between object lookup mechanisms is 
> probably the biggest reason for JXTG's complexity.
Sure. But JXTG mainly suffers from the lack of extendibility. You can
see a lot of JXTG hacks in cocoon samples even right now.

There are two hacks that I use for JXTG all the time:
- use a external function (static method, flowscript function) in jxtg
that accepts a bean and xml consumer to generate sax events:
   ${Packages.com.something.Utils.prettyPrint( bean.value,
cocoon.consumer )}

- use a hash map and a set of macros to handle dynamic tags:
> <jx:set var="tags" value="${java.util.HashMap()}"/>
> <jx:macro name="dynamic-tag">
>     <jx:parameter name="id"/>
>     <jx:set var="ignored" value="${tags.put(id, macro.body)}"/>
> </jx:macro>

declare a dynamic tag:
> <dynamic-tag id="resultRow">
>     <td><a href="/nTer/projectDetails.do?projectId=${project.id}"><b>${project.name}</b></a></td>
>     <td>
>         <radeox-string value="${project.description}"/>
>     </td>
> </dynamic-tag>

later on use it in macro for example:
> <jx:forEach var="${item}" items="${items}" varStatus="status">
> 	<colouredRow rowNo="${status.index}" odd="floralwhite" even="#eaeaea">
> 		<jx:eval select="${tags.resultRow}"/>
> 	</colouredRow>
> </jx:forEach>

*I just found that we promote this hack on cocoon site*:
http://cocoon.apache.org/2.1/userdocs/flow/jxtemplate.html#eval

Both cases indicate that JXTG allows for a great amount of dirty tricks. 
The users will use them as long as we do not give them a better tool.

> I'm a big proponent of "make the better decision easier than the worse 
> decision."  Quick and dirty, ESQL works.  For limited obscure cases, 
> ESQL is great.  But the better decision would be to use object mapping 
> (Hibernate, JDO, et al.) and CForms in Flow.  Tag libraries seem to me 
> to promote an excessive shift of logic to the templating language 
> instead of in the backend data processing architecture.  Not forcing but 
> encouraging people to move as much logic as possible to Flow seems like 
> it would allow more flexibility while remaining more maintainable and 
> clear.  "Clear" not in the sense that the individual template and app is 
> understandable but that the hundredth template and app is understandable.
The user should be informed of the danger of mixing the concerns. But 
it's his/hers choice to break the rules. Cocoon should promote good 
practices not enforce them.

>> For those of us who use CForms it is very convenient to be able to use 
>> the template tags together with JXTG constructs.
>>
>> So we need a template generator with three sets of tags "jx:", "esql:" 
>> and "ft:" thats it.
>>
>> We also discused how to use a separate conversion layer to remove the 
>> need for formating constructs from the template layer.
> 
> 
> I'm not following.  If you are using CForms (and thus Flow), why would 
> you make template calls to a database?  Why not do the persistence logic 
> in Flow right next to where you are sending and receiving form data?  
>  From a logical point of view, I would think that these would be better 
> served next to one another.
The whole conversation got totally polluted with ESQL. Templating is 
very useful for CForms dependent on model data.  I.e. render the delete 
button only if model says you are allowed to invoke that action.

<snip what="even more esql pros and cons"/>

my regards
-- 
Leszek Gawron                                      lgawron@mobilebox.pl
Project Manager                                    MobileBox sp. z o.o.
+48 (61) 855 06 67                              http://www.mobilebox.pl
mobile: +48 (501) 720 812                       fax: +48 (61) 853 29 65

Re: [Design] JXTG 2.0 (Just say you're sorry!)

[Torsten Curdt <tc...@apache.org>]

<snip type="a very interesting discussion"/>

..now let me chime in

Having spent quite some time with XSP and ESQL
(and friends) I have to admit: XSP can become a
PITA. But usually because of taglibs (...as
already mentioned in the thread)

The very basic XSP is not much more but using
java as your templating language. Well, the
thing is java and XML just don't mix very well.
...but that's just a question of the syntax.

Yes... I do think that taglibs are the evil
of XSP. The compilation part ...not nice but
more or less depends on how you deal with it.

The good and the bad is that XSP gives the
full power of java. ...full access to every-
thing available in the jvm.

> But is ESQL is the best choice?  Data input becomes a big deal when
> you use ESQL for output and then a stream generator combined with 
> transformation combined with sqltransformer combined with...
> 
> I'm a big proponent of "make the better decision easier than the
> worse decision."  Quick and dirty, ESQL works.  For limited obscure
> cases, ESQL is great.

Hm... I am wondering what you mean by the big
deal of the input. Fact is that ESQL has some
very unique features. ..that are needed in not
so obscure cases.

I might not be up-to-date with object mapping
frameworks but even if you say that they all
support query time row limitations ...so tables
of a million+ rows are still page-able without
having the data shuffled across the network by
the stupid JDBC driver implementation ...there
still is the question if *you* do the database
design. Mapping to a bad designed database can
be also be no big fun.

Anyway what I was trying to say ...sometimes
SQL can be a very concise syntax for what your
framework might make even more complicated.
Sometimes...

...it just depends!


I think a much bigger problem of ESQL (and also
the SQLTransformer) is the time when the database
processing is happening. ...just too late.
Inside the view ...to bring the MVC up again.

It should better be inside the controller.
Which would be most likely flow. But noone
wants to pollute the flow with SQL statements
(does someone)? ...that's the real beauty of
flow with a relational mapping IMHO.

my 2 cents
--
Torsten