[RT] Formatting and Macros: A tangent on the template/taglib discussion

["Roy G. Biv" <mi...@pcextremist.com>]

Here's where I'm at on templates.  Feel free to critique.

Goals (in order of importance):
 1. Get data from an object model
 2. Minimize or eliminate programmatic logic inside the template
 3. Make as simple to read/write templates as possible
 4. Give feedback on all possible errors
 5. Make the data presentable -- follow established Java data formatting 
rules where possible
 6. Speed
 7. Allow extensibility without altering core code

Things to avoid:
 1. Making a turing complete programming language out of the template 
language
 2. Avoid tags that simply duplicate programmatic language structures 
(eg. <if> <else> <forEach>)
 3. Second-guessing the data model (eg. getInt, getFloat, getString)
 4. Programmatic tag libraries for macros (ie. Just use STX, stx:include 
and aggregate tranforms)

While I doubt my goals will draw much criticism, the latter group may.  
In my opinion, once you fall into if/else/forEach land, you've just 
rewritten XSP with attribute markers instead of <xsp:logic> markers.  
I'm probably a broken record here, but there is no difference between 
<if test="x"/> and if(x){}.  If the templating process inevitably falls 
to them, assumptions about the template tools or that template authors 
must write them must be reconsidered.  Simplicity and being terse are 
key.  If an operation can be done with one attribute what would require 
three attributes (or one attribute with three directives) for 
Turing-completeness, pick the single, simple attribute that is not 
Turing-complete.

In addition, I'm wondering what folks think of the following:

-----------------------------
Template Typing/Formatting
-----------------------------

Data structures are largely uniform.  If I query a bean for a user's 
age, it is highly unlikely that sometimes I will receive an integer and 
other times receive string or a date during the same data iteration.  So 
in the efforts toward template simplification, how do we feel about 
these two items?

1. No explicit data typing by default (eg. 
${user/sallary::#,###.00'/yr'} instead of something like 
${formatNumber(user/sallary, '#,###.00\'/yr\'')}) or the current crop of 
<formatNumber/> and its ilk.

rationale: promotes the use of appropriate data types by the programmer 
and relieves the template author of needing to know about typing.  The 
persistence mechanism most likely has a type specified (especially if a 
relational database is in use) and objects in Flow as well.  Why force 
restatement in the template byt the person arguably least likely to make 
an informed decision on the subject?  Just pass a format parameter and 
be done with it.  If the format parameter doesn't fit, report the error 
(and location in the template).

2. Use and extension of the java.text.Format hierarchy for data formatting.

rationale: code reuse is maximized and avoids the proliferation of 
Cocoon-only APIs.

As was mentioned before, there are only so many formatting variants 
needed.  I think this covers the bases:

 boolean: a selection from two possibilities eg. true/false, yes/no, on/off.
 integer (byte, short, int, long): separator placement, percentages, 
currency (precision with decimals by division), other bases (hex, octal, 
binary)
 decimal (float, double): separator placement, percentages, currency, 
decimal precision
 date/time: basically the SimpleDateFormat syntax featureset
 strings (including char and char[]): substring, lowercase/uppercase (Is 
this a CSS concern?), xml fragments
 unhandled objects: Object.toString() should cover this

Also, integer, decimal, and date/time parameters would have to handle 
locale.  (Or would a template-wide locale setting in the sitemap be more 
appropriate?)

Missing(?) items are formatters for java.math.BigInteger and 
java.math.BigDecimal.  Also perhaps support for InputStreams/Readers for 
larger content (treated as a string or xml InputSource I would assume).  
Also a mechanism to add/register new formatters.

By factoring out this code concern and relying upon java.text.Format 
implementations, the amount of Cocoon-specific code could be reduced 
while referencing "standard" formatting parameters.

Getting the appropriate format could be accomplished with a 
FormatFactory (for lack of a better name) with the following interface:

    interface FormatFactory {
        public java.text.Format getFormat (String params, Locale 
defaultLocale);
        public java.text.Format getFormat (String params, Locale 
defaultLocale, Object value);
    }

The first is called to see if the Format can be determined by the 
parameters.  The second is called when no formatter was returned from 
the first so that the Object's type determines the formatter.  As I said 
earlier, the types are usually uniform in any given data structure.  
Once the type is determined and the format set once, further iterations 
could skip this type check and just use the formatter.  If there is no 
appropriate formatter for the type, a default formatter that simply 
calls .toString() would be used.

With this, formatting concerns are handled in code that's not intimately 
tied to (and easily separated from) any particular sitemap component and 
can reuse code not specifically made for Cocoon.


-----------------------------
Macros
-----------------------------

I strongly encourage the use of STX stylesheets for this.  Let's say you 
have the following markup from the JXTG samples:

  <table>
    <tablerows list="${greatlakes}" color="blue"/>
  </table>

I'd prefer using an alternate namespace, but I digress.  You make a STX 
stylesheet with the relevant protion:

  <stx:template match="tablerows">
    <jx:forEach var="item" items="{@list}">
      <tr><td bgcolor="{@color}">${item}</td></tr>
    </jx:forEach>
  </stx:template>

     or with attribute syntax

  <stx:template match="tablerows">
    <tr jx:forEach="item in {@list}"><td 
bgcolor="{@color}">${item}</td></tr>
  </stx:template>

This is what is happening after all whether in Java or not, isn't it?  
Anyone want to tell me that this is harder and requires more knowledge 
than writing, compiling and installing a Java class?  With this, macro 
languages can be far more dynamic and immediate to the problems at hand; 
syntactic sugar and simplification with no fear of side effects.  Now, 
group all your macros together into a file called "form-macros.stx".  
Put any other macro groups in their own file as well.  Then you combine 
them all into a file called macros.stx (or just have a directory 
generator run on a directory full of macro definition files and 
dynamically generate the aggregating stylesheet):

  <stx:transform  version="1.0" 
xmlns:stx="http://stx.sourceforge.net/2002/ns">
    <stx:include src="template-macros.stx"/>
    <stx:include src="form-macros.stx"/>
    <stx:include src="john-macros.stx"/>
    <stx:include src="mary-macros.stx"/>
  </stx:transform>

and processed by

  <map:transform type="stx" src="macros.stx"/>

     or

  <map:transformer name="macro">
    <map:transform type="stx" src="macros.stx"/>
  </map:transformer>

     followed by

  <map:transform type="macro"/>

(By the way, have I mentioned lately that I love the concept of virtual 
components?)

There you have your modular, maintainable macro library.  STX when used 
in this manner is sufficiently fast, not overly complex, uses very 
little memory (SAX-based), and is highly cacheable.  This is why I don't 
like the idea of this type of thing going into full, programmatic 
logic.  A Java interface behind macros strikes me as an overly complex 
solution and not as easy to edit/maintain as the stylesheet method.

-----------------------------

I most definitely see macros and formatting/data injection to be two 
separate issues that do not require a unified interface for a solution.  
Formatting needs to happen when the data is dropped in because the type 
of the data is a large determining factor of how it's 
formatted/processed/streamed.  Macros on the other hand are syntactic 
sugar made to reduce the amount of typing one must do.  There is no 
functional requirement for the macro expansion and the data 
injection/formatting to be combined into a single step from the 
implementation point of view -- only from the template author's point of 
view.

This leaves the sitemap component responsible for templates simpler, 
easier to maintain, and focused on just the concerns in its scope -- 
iterating data structures and injecting data.  ...with the help of some 
standard data formatters.  No muss.  No fuss.  No flexibility syndrome.  
Clearly defined roles.

- Miles Elam

Re: [RT] Formatting and Macros: A tangent on the template/taglib discussion

["Roy G. Biv" <mi...@pcextremist.com>]

Thank you for the reply.  I realize that I'm trying your patience.  I 
appreciate you bearing with me.

Daniel Fagerstrom wrote:

> Roy G. Biv wrote:
>
>> Goals (in order of importance):
>> 1. Get data from an object model
>> 2. Minimize or eliminate programmatic logic inside the template
>> 3. Make as simple to read/write templates as possible
>> 4. Give feedback on all possible errors
>> 5. Make the data presentable -- follow established Java data 
>> formatting rules where possible
>> 6. Speed
>> 7. Allow extensibility without altering core code
>
> Seem reasonable. Point 4. seem complicated, care to expand on this. 
> Concerning point 7., I'm curious, wasn't you in strong oposition 
> against extensibillity?

Point 4: JXTG does it already.  Basically it amounts to saying what went 
wrong and what line/column the error in the template occurred on.

Point 7: I was referring to data formatters.  Sorry about that.

>> 1. Making a turing complete programming language out of the template 
>> language
>
> Things to avoid:
>
> The important point is not Turing compatibility, BTW I woudn't think 
> that the attribute language I proposed it Turing compatible.
>
> The important things is IMO, that the language is side effect free and 
> and has access to the things but not more. What that means is an 
> important point to discuss. It is also important that the directives 
> are so well designed so that it is easy to perform common tasks.

Agreed.

>> 2. Avoid tags that simply duplicate programmatic language structures 
>> (eg. <if> <else> <forEach>)
>
> Is this a syntax or semantics question? I.e. does it mean that you 
> think they are ok as attributes but not as tags or does it mean that 
> you will rule out such constructs completely?

It means I would try to stop thinking as a programmer and focus on the 
problem.  I know that seems in contradiction on its face since no matter 
what, we're dealing with a computer.  But I know for myself, I've spent 
so much time with C/C++/Java/Perl/Javascript that if, else, foreach, 
while, switch, etc. are all part of my natural language.  I like many 
many others on this list think in code.  The distinction between a class 
constant and an instance variable is clear and distinct on an almost 
instinctive level due to years of repetition.  "Singleton" and "facade" 
are in our daily vocabulary.

*We as programmers are odd.*

When designing template languages for example, I try to think as someone 
who hasn't taken an introductory programming or data structures class in 
college.  I try to think of my friends who think of if, else, while, 
etc. as just a few arbitrary words out of thousands in the English 
language.  Then add only what logic is necessary to get to a 
well-defined target.  Since if, else and foreach are all general 
programming directives, I consider their presence in a template language 
suspect until proven otherwise necessary.  Nothing nefarious or 
complex.  Just a reflex so that I can empathize with the target audience.

>> 4. Programmatic tag libraries for macros (ie. Just use STX, 
>> stx:include and aggregate tranforms)
>
> As said before I don't share your love for STX, I find it rather cool 
> for applications where I need to do simple things for huge amount of 
> repetetive data, but otherwise its far to complicated for my taste.

I guess I'm coming from the point of view of the samples for JXTG -- not 
the most complex material to work with.  Then again, that's the type of 
thing that I think of when I hear the term "template macro".  With the 
calendar example, I found the complexity to be the bean and the raw 
template directives necessary to iterate that bean.  The macro expansion 
portion never occured to me to be anything but the most trivial portion 
of the equation.

>> While I doubt my goals will draw much criticism, the latter group 
>> may.  In my opinion, once you fall into if/else/forEach land, you've 
>> just rewritten XSP with attribute markers instead of <xsp:logic> 
>> markers.  I'm probably a broken record here, but there is no 
>> difference between <if test="x"/> and if(x){}.
>
> Yes you are, so I try to avoid repeating my answer ;)

Haha!  Sorry about that.

>> In addition, I'm wondering what folks think of the following:
>>
>> -----------------------------
>> Template Typing/Formatting
>> -----------------------------
>
> Yes, what do we think? Take a look at: [RT] Attribute Rendering and 
> CForms Convertors, 
> http://marc.theaimsgroup.com/?t=109941988300003&r=1&w=2
> all the 78 posts are not about that but many are ;)

Ahhhh...  This makes it clear for me.  This also explains why I missed 
the discussion.  I may just be slow, but I never considered that email 
subject line to be about JXTG template replacements.  I have been in 
general quite pleased with CForm work thus far and saw no need to get 
involved with implementation discussions about it.  I'm working my way 
through the discussion now to catch up.

> Miles, it seem that you propose things that are very close to things 
> that I and others have discussed the design for in previous e-mails. 
> If there are things that I fail to explain in a clear way it would be 
> better if you just asked what I mean instead of assuming the worst.

Reading and catching up now.

- Miles Elam

Re: [RT] Formatting and Macros: A tangent on the template/taglib discussion

[Daniel Fagerstrom <da...@nada.kth.se>]

Roy G. Biv wrote:
> Here's where I'm at on templates.  Feel free to critique.
> 
> Goals (in order of importance):
> 1. Get data from an object model
> 2. Minimize or eliminate programmatic logic inside the template
> 3. Make as simple to read/write templates as possible
> 4. Give feedback on all possible errors
> 5. Make the data presentable -- follow established Java data formatting 
> rules where possible
> 6. Speed
> 7. Allow extensibility without altering core code

Seem reasonable. Point 4. seem complicated, care to expand on this. 
Concerning point 7., I'm curious, wasn't you in strong oposition against 
extensibillity?

> Things to avoid:
> 1. Making a turing complete programming language out of the template 
> language

The important point is not Turing compatibility, BTW I woudn't think 
that the attribute language I proposed it Turing compatible.

The important things is IMO, that the language is side effect free and 
and has access to the things but not more. What that means is an 
important point to discuss. It is also important that the directives are 
so well designed so that it is easy to perform common tasks.

> 2. Avoid tags that simply duplicate programmatic language structures 
> (eg. <if> <else> <forEach>)

Is this a syntax or semantics question? I.e. does it mean that you think 
they are ok as attributes but not as tags or does it mean that you will 
rule out such constructs completely?

> 3. Second-guessing the data model (eg. getInt, getFloat, getString)

Agree.

> 4. Programmatic tag libraries for macros (ie. Just use STX, stx:include 
> and aggregate tranforms)

As said before I don't share your love for STX, I find it rather cool 
for applications where I need to do simple things for huge amount of 
repetetive data, but otherwise its far to complicated for my taste.

> While I doubt my goals will draw much criticism, the latter group may.  
> In my opinion, once you fall into if/else/forEach land, you've just 
> rewritten XSP with attribute markers instead of <xsp:logic> markers.  
> I'm probably a broken record here, but there is no difference between 
> <if test="x"/> and if(x){}.

Yes you are, so I try to avoid repeating my answer ;)

> In addition, I'm wondering what folks think of the following:
> 
> -----------------------------
> Template Typing/Formatting
> -----------------------------

Yes, what do we think? Take a look at: [RT] Attribute Rendering and 
CForms Convertors, http://marc.theaimsgroup.com/?t=109941988300003&r=1&w=2
all the 78 posts are not about that but many are ;)

<snip/>
> With this, formatting concerns are handled in code that's not intimately 
> tied to (and easily separated from) any particular sitemap component and 
> can reuse code not specifically made for Cocoon.

Yes, we are at the same page as you can see in the above thread. We 
found a solution that separated concerns some what better, but it's 
quite similar. Jonas have implemented a prototype of it, but its not 
included in svn yet.

> -----------------------------
> Macros
> -----------------------------
> 
> I strongly encourage the use of STX stylesheets for this.  Let's say you 
> have the following markup from the JXTG samples:
> 
>  <table>
>    <tablerows list="${greatlakes}" color="blue"/>
>  </table>
> 
> I'd prefer using an alternate namespace, but I digress.  You make a STX 
> stylesheet with the relevant protion:
> 
>  <stx:template match="tablerows">
>    <jx:forEach var="item" items="{@list}">
>      <tr><td bgcolor="{@color}">${item}</td></tr>
>    </jx:forEach>
>  </stx:template>
> 
>     or with attribute syntax
> 
>  <stx:template match="tablerows">
>    <tr jx:forEach="item in {@list}"><td 
> bgcolor="{@color}">${item}</td></tr>
>  </stx:template>

Why not just:

<tr do="macro(tablerows,list,color);forEach($list)">
   <td bgcolor="{$color}">${item}</td>
</tr>

or

<jx:macro name="tablerows">
   <jx:parameter name="list"/>
   <jx:parameter name="color"/>
   <jx:forEach var="item" items="{$list}">
     <tr><td bgcolor="{$color}">${item}</td></tr>
   </jx:forEach>
<jx:macro>

why do you want to use an external language for the macro expansion, it 
should be enough with learning the template language for writing macros.

> This is what is happening after all whether in Java or not, isn't it?  
> Anyone want to tell me that this is harder and requires more knowledge 
> than writing, compiling and installing a Java class?

It seem to me that you believe that I or someone else have sugested that 
  macros like tablerow should be written in Java. If that is the case I 
can understand your frustration. But I cannot understand what you have 
got that idea from. I have most certainly never sugested anything even 
close to it. I would find it horible to write my webapps that way.

Java is needed for writing "if", "forEach" and a number of other core 
constructs, nothing else.

> With this, macro 
> languages can be far more dynamic and immediate to the problems at hand; 
> syntactic sugar and simplification with no fear of side effects.

> There you have your modular, maintainable macro library.  STX when used 
> in this manner is sufficiently fast, not overly complex, uses very 
> little memory (SAX-based), and is highly cacheable.  This is why I don't 
> like the idea of this type of thing going into full, programmatic 
> logic.  A Java interface behind macros strikes me as an overly complex 
> solution and not as easy to edit/maintain as the stylesheet method.

<snip/>

Miles, it seem that you propose things that are very close to things 
that I and others have discussed the design for in previous e-mails. If 
there are things that I fail to explain in a clear way it would be 
better if you just asked what I mean instead of assuming the worst.

/Daniel

Re: [RT] Formatting and Macros: A tangent on the template/taglib discussion

["Roy G. Biv" <mi...@pcextremist.com>]

Stefano Mazzocchi wrote:

> 2) at the end, the template is not a file, but the result of a 
> pipeline fragment. In short, the template that DW designers will have 
> to use is a pipeline view, not a file on a disk.

How so?  The preliminary step(s) before data injection include such 
items as macros before expansion.  In short, the file on disk looks 
conspicuously like the template generator that does macros internally.  
I would absolutely expect the DW designer to be referencing the macros 
pre-expansion, not post-.

Whether it's

  <map:generate type="jxtemplate" src="template.jx"/>

or

  <map:generate type="file" src="template.jx"/>
  <map:transform type="stx" src="macros.stx"/>
  <map:transform type="template"/>

> This means that in order to allow DW designers to edit their pages, 
> you have to create a special section of the website, hopefully webdav 
> enabled, so that they can open/edit/save those "virtual files"... 
> also, you need the ability for your first transformation stage to be 
> reversible.

It's the origin template (template.jx) that would be edited in DW, not 
the output of the macro transformer or data injection.  If you don't 
require reversal after data injection to get to HTML for the generator, 
why would you require it in this transformation model?  "template.jx" is 
the editable source in either case.

I may be working with a Simplified DocBook variant (and have to deal 
with some WebDAV trickery for editing), but nothing here requires it any 
more than if I created the file with JXTG to produce said Simplified 
DocBook variant.  XSLT transformation to HTML is the precursor to 
reversal, not a data injection transformer.

- Miles Elam

P.S.  I use the term "data injection", but if that is turning some folks 
off, is there a preferred term that I should use on this list instead?

Re: [RT] Formatting and Macros: A tangent on the template/taglib discussion

[Stefano Mazzocchi <st...@apache.org>]

Roy G. Biv wrote:
> Here's where I'm at on templates.  Feel free to critique.

[snip]

I have to admit there is interesting value in your "everything is a 
transformer" position. There are two things I dislike:

1) your pipelines are big and complex

                       beans
                         |
                         v
  file -> xslt/stx -> template -> xslt/stx -> html

vs.

  beans
    |
    v
  template -> xslt/stx -> html

But it's true that your pipeline can be rewritten using a virtual 
generator, so

  template := file -> xslt/stx -> template-trans

so that the above pipeline can be the exact same, therefore it's not 
such a big problem.

2) at the end, the template is not a file, but the result of a pipeline 
fragment. In short, the template that DW designers will have to use is a 
pipeline view, not a file on a disk.

This means that in order to allow DW designers to edit their pages, you 
have to create a special section of the website, hopefully webdav 
enabled, so that they can open/edit/save those "virtual files"... also, 
you need the ability for your first transformation stage to be reversible.

I've tried the above a few times in the past and without much success I 
have to say.

In short, the need for people to use existing tools to edit the 
templates forces you to have them as files, which forces you to have the 
template system as a generator (unless, you can come up with some clever 
general purpose yet reversible transformation stage since I couldn't and 
convince people to use webdav-enabled URL spaces just to have their 
designers work on the templates).

-- 
Stefano.