Jsp scripting elements revisited

[ru...@us.ibm.com]

I'm sure that elements of what I am about to post belong on jsp-tfaq (and
even possibly cocoon-dev), but being adverse to cross posting, and knowing
that an appropriate cross section of the people follow this mailing list,
I've chosen tomcat-dev.

Danno wrote:
> Another interesting point: a well known and heavily used applicaiton
> server has in the past had some bone headed design errors in their
> servlet engine in the past.  So given that if the same vendor or
> another application server provider implemented a legitamatly ambiguous
> point in a manner that is later clarified as incorrect is not the
> slightest bit suprising and shows that they still have non-clarvoiant
> people writing the code.  In fact that would include just about any
> software vendor except those who write code "for entertainment purposes
> only."

I'm not sure what is behind the obviously strong feelings on this subject.
The best way to get beyond this is to get everything out on the table.  In
part I of this post, I will argue for a but more consideration and
discussion before these types of changes are committed.  Part II will for
the first time lay out why this particular function is so important to me.
Part III will simply attempt to bring this together into an action plan

 - - - - - - - - - - - - - - - - - - - - - - - - -

Part I - where we are

A few other "interesting points" in order to foster discussion.  There
apparently was ambiguous prose which was interpreted in a way that was
later deemed to be incorrect.  In the case of ambiguity, presumably one or
more people look to the reference implementation for guidance.
Furthermore, it is quite likely that one or more products either have
already, or are contemplating in the near future, the actual inclusion of
the Jasper code in their product.  This is something that is not only
allowed by the Apache license, but something that we should actively
encourage.

Simply put, those people and products are victims, pure and simple.  As are
their customers.  If we do not keep this in the forefront of our minds at
all times, we stand a significant danger of will losing their loyalty.  In
my mind, this is a far greater danger to the status of JSP as a
multi-vendor standard than the presence of minor and arguably allowable
extensions (or at least, not specifically prohibited) in a reference
implementation.

Please don't misconstrue my previous statement.  I'm not suggesting that we
should actively seek to introduce extensions in the reference
implementation - far from it.  But when we find ourselves in a situation
such as we are in today, we should seek the right balance between where we
have been, how it may affect the community, and where we expect to evolve
this specification towards.

In short, I would argue for a bit more "due process" between the discussion
of a clarification of a relatively obscure and obviously misunderstood
portion of the spec and the radical elimination of all support for this
function.

 - - - - - - - - - - - - - - - - - - - - - - - - -

Part II - where we want to be

I've had a rough idea of where I would like to evolve Jasper towards, but
prior to the remedial education of tag extensions I got yesterday (thanks
Eduardo!) and the reading I did on the flight back home, I didn't have an
adequate vocabulary in which to express these ideas.  I'm now feel somewhat
closer to being up to the task, so here goes.

Prior to the addition of tag extensions to the JSP spec, there were a set
of predefined JSP tags.  The support for these tags are "hard wired" into
Jasper today.  Tag extensions open up the possibility for third parties to
define additional tags which can be used in a very similar manner as the
predefined tags.  Looking at the Jasper code, the support for these tags is
separate and distinct from the predefined tags.

I'd like to evolve Jasper towards a place where all tag support is unified.
For starters, I'd like to see a TLD defined for the current set of JSP
tags.  Then I would like to see the elimination of the the special case
parsing for these tags eliminated as much as humanly possible.  I'd also
like to see the special syntaxes (e.g., <% %>) be unified as well, in
particular becoming nothing more than a convenient shorthand for an
equivalent predefined tag.  One could also imagine an implicit <jsp:block>
surrounding the page, and furthermore, one could define the page directives
in terms of how they affect these transformations (e.g., the page language
attribute would be transferred into attributes on the implicit
jsp:scriptlets generated by regions on that page, but would not affect the
processing of included pages or the subsequent recursive processing of tag
output).

Trying to get this to work is likely to flush out some dark corners of the
specification, and stress portions of the implementation.  Particular areas
of concern are nested tags and dealing with statement fragments in
scriptlets.  This is all goodness.  Once completed, we will have a much
greater confidence that both the spec and implementation are suitable for
the purpose intended.

Finally, it is conceivable that the implementation of these tags could
possibly be reused by other JSP engines, or at least inspire development of
corresponding support in other publishing frameworks.  This could increase
the ability of people to migrate their pages more easily from one
technology or standard to another.  Whatever allegiance one might feel to a
particular specification - the level of openness is always goodness, and is
exactly what I have been working towards with PHP, Perl, and other
integration efforts.

 - - - - - - - - - - - - - - - - - - - - - - - - -

Part III - a plan to get there

For starters, I would like to see tags originally introduced in order to
support XML reintroduced into the code base of Jasper, for both tactical
and strategic reasons as outlined above.  In parallel, the TLD for the "JSP
standard tag library" should be developed.  This would benefit greatly by
people by contributions from those who have a deeply understanding of the
spec.  In fact, I would argue that it would make more sense to incorporate
the result into the subsequent revisions of the spec instead of merely into
Jakarta.

Once both are completed, Jasper's special case parsing logic should be
changed to incorporate the new TLD and implement the majority of the
standard tags in terms of more primitive tags, much in the way that the
majority of the standard Java packages are themselves written in Java.  I
believe that this can be done in a very incremental fashion.  Furthermore,
the spec and the implementation should "co-evolve" for a period of time as
some of the experience gained in implementing some of the stickier issues
mentioned above may prove invaluable in helping form the correct
specification for the desired behavior.

Interestingly enough, if you look ahead to logical conclusion of this
proposal, the likely conclusion is pretty much the opposite of the current
situation: there will likely be a small kernel of built-in functions
(including scriptlet), with the remainder moved out and into predefined
extensions to the base.

- Sam Ruby

Re: Jsp scripting elements revisited

[Danno Ferrin <sh...@earthlink.net>]

Ok, perhaps the quote you are basing off on wasn't written very well, it was
ment to be subtly humorus.  The bug I was referring to was a bug about three
years ago.  Yes, that was the Servlet 1.0 days, and had to do with named
servlets v.s. calss servlets.  Then I went to how the intermixing of XML
tags was ambiguous, and since programmers aren't psychic then a vendor's
programmers could not know that after they would write their JSP parser that
this point would be clarified.  Programmers arn't psychic, which was the
subtle humor there.  If I do it again I will warn you with XML elemenets, of
course they will be in a namespace within a validated document (where the
namespeaced elements are not in the DTD... whoops there I go agian, <humor
level='subtle' quality='poor'/>).  Plus I have spent most of the week
getting stuff moved and starting a new job, I don't have the same amount of
time to hone my humor.  Now for something legitamatly serious.



rubys@us.ibm.com:

>
>
> I'm sure that elements of what I am about to post belong on jsp-tfaq (and
> even possibly cocoon-dev), but being adverse to cross posting, and knowing
> that an appropriate cross section of the people follow this mailing list,
> I've chosen tomcat-dev.
>
> Danno wrote:
> > Another interesting point: a well known and heavily used applicaiton
> > server has in the past had some bone headed design errors in their
> > servlet engine in the past.  So given that if the same vendor or
> > another application server provider implemented a legitamatly ambiguous
> > point in a manner that is later clarified as incorrect is not the
> > slightest bit suprising and shows that they still have non-clarvoiant
> > people writing the code.  In fact that would include just about any
> > software vendor except those who write code "for entertainment purposes
> > only."


> I'm not sure what is behind the obviously strong feelings on this subject.
> The best way to get beyond this is to get everything out on the table.  In
> part I of this post, I will argue for a but more consideration and
> discussion before these types of changes are committed.  Part II will for
> the first time lay out why this particular function is so important to me.
> Part III will simply attempt to bring this together into an action plan

Actually I have no hard feelings on it.  When I first posted the patch and
it got -1ed I was fine and I posted why I though they should be removed in
the patch.  Well when eduardo got back from vacation he picked up on that
thread and said that this support shouldn't be there.  In response to
another thread about my ramblings to try and get the XML support something
close to what the spec dictates he said I should remove the tags ASAP and to
hold off on the support of the XML as a JSP support because he mentioned
that he totally glossed over that section of the spec when revising it.  So
I did it because the Spec God spoke.

>  - - - - - - - - - - - - - - - - - - - - - - - - -
>
> Part I - where we are
>
> A few other "interesting points" in order to foster discussion.  There
> apparently was ambiguous prose which was interpreted in a way that was
> later deemed to be incorrect.  In the case of ambiguity, presumably one or
> more people look to the reference implementation for guidance.

When I wrote my own JSP engine that is exactly what I did, and is also why I
included the XML tags as scripting elements.  In fact up until about a month
ago there was notation from a non-public spec in one of the demos and in the
code.  Clearly the XML portion of the spec was not anything resembling a
priority to the J2EE team, and I see no reason why it should have been given
all of the other things they need to integrate with in the J2EE spec.

> Furthermore, it is quite likely that one or more products either have
> already, or are contemplating in the near future, the actual inclusion of
> the Jasper code in their product.  This is something that is not only
> allowed by the Apache license, but something that we should actively
> encourage.

And I am sure that the jakarta project is ruining more than one business
plan out there.  I wouldn't be supprised if that and IP issues was why JRun
agreed to be bought by Allaire.

> Simply put, those people and products are victims, pure and simple.  As
are
> their customers.  If we do not keep this in the forefront of our minds at
> all times, we stand a significant danger of will losing their loyalty.  In
> my mind, this is a far greater danger to the status of JSP as a
> multi-vendor standard than the presence of minor and arguably allowable
> extensions (or at least, not specifically prohibited) in a reference
> implementation.

I have no problem with that, but when the preson who wrote the spec (or is
the most accountable party in that regard) says we are doing something wrong
then I tend to listen.

> Please don't misconstrue my previous statement.  I'm not suggesting that
we
> should actively seek to introduce extensions in the reference
> implementation - far from it.  But when we find ourselves in a situation
> such as we are in today, we should seek the right balance between where we
> have been, how it may affect the community, and where we expect to evolve
> this specification towards.

<humor> Now if your e-mail were rubys@microsoft.com then you _would_ be
encouraging the introduction of extensions in the refrence implementation,
right?  (That hasn't _ever_ happend, now has it?)</humor>

> In short, I would argue for a bit more "due process" between the
discussion
> of a clarification of a relatively obscure and obviously misunderstood
> portion of the spec and the radical elimination of all support for this
> function.

Don't take this the wrong way, but is there any demonstratable harm to what
was done?  (removing the support)  We could add an option that will use the
tags but only after presenting the admin or author with a really annoying
deprecation/not in the spec warning.  Kind of what Microsoft had to do for a
while to their java compiler.  But since it is something that was almost
always outside the scope of the spec and the notion of using them was
introduced by not scrubbing down the demos I really don't see why this would
affect many people.  Of course Eduard/o could just fake us all and add it
into the jsp.next spec. ;)

>  - - - - - - - - - - - - - - - - - - - - - - - - -
>
> Part II - where we want to be
>
> I've had a rough idea of where I would like to evolve Jasper towards, but
> prior to the remedial education of tag extensions I got yesterday (thanks
> Eduardo!) and the reading I did on the flight back home, I didn't have an
> adequate vocabulary in which to express these ideas.  I'm now feel
somewhat
> closer to being up to the task, so here goes.
>
> Prior to the addition of tag extensions to the JSP spec, there were a set
> of predefined JSP tags.  The support for these tags are "hard wired" into
> Jasper today.  Tag extensions open up the possibility for third parties to
> define additional tags which can be used in a very similar manner as the
> predefined tags.  Looking at the Jasper code, the support for these tags
is
> separate and distinct from the predefined tags.
>
> I'd like to evolve Jasper towards a place where all tag support is
unified.
> For starters, I'd like to see a TLD defined for the current set of JSP
> tags.  Then I would like to see the elimination of the the special case

Actually in the 1.1pr1 spec and before the first jakarta code drop (It's
location escapes me) there was some mention that the RI was going to do a
TLD for the standard JSP actions, just like you are suggesting.  The tag
library API to me appears to support it.  My guess is that the writing of
the J2EE RI used up all of the resourses that would have been needed to do
it.  And I can't find mention of it in later specs.  As a teaching tool to
show other programers how a TLD works I think it would be great, because if
they understand how the standard actions work then they can look in  the
standard action tag handlers to see how it is done. However one of the
cavets in the 1.1PR1 spec was that it may lead to performance degredation.
WRT the bean code I am almost certian that we will se some degredation with
set/getProperty and useBean becuase of the reflection that can be optimized
out at compile time.


> parsing for these tags eliminated as much as humanly possible.  I'd also
> like to see the special syntaxes (e.g., <% %>) be unified as well, in
> particular becoming nothing more than a convenient shorthand for an
> equivalent predefined tag.  One could also imagine an implicit <jsp:block>
> surrounding the page, and furthermore, one could define the page
directives
> in terms of how they affect these transformations (e.g., the page language
> attribute would be transferred into attributes on the implicit
> jsp:scriptlets generated by regions on that page, but would not affect the
> processing of included pages or the subsequent recursive processing of tag
> output).

Using the taglibrary API as it stands doing so would be a monumental task
that would include integrating or basically writing a java compiler since
the blocks can start in one tags and stop in the other.  I'm not saying it
cannot be done, but it is a whole lot of effort that is best spent in other
places, such as adding ECMA 262/NetRexx/JPython support in the language page
directive (isn't that why you came on board sam?)  Perhaps we could convince
some sadistic CS professor to make the scripting tags as a TLD a senior
project to some unsuspecting innocents.  But the tags would have to be
written on a per-language basis, adding JPython would require re-writing the
tags.  The current setup of the taglib API keeps that issue completely
compartmentatlized away from the tag libraries.  As long as the languages
fully use the JVM any part can be written in any language as long as the JVM
protocols are kept.

> Trying to get this to work is likely to flush out some dark corners of the
> specification, and stress portions of the implementation.  Particular
areas
> of concern are nested tags and dealing with statement fragments in
> scriptlets.  This is all goodness.  Once completed, we will have a much
> greater confidence that both the spec and implementation are suitable for
> the purpose intended.

One of the current "features" of the spec is that it is rather language
agnotstic, as long as it is the Java2 platform, jvm, and libraries.  That is
why nested scriptlet tags are handled as special cases, becuase it should be
seperated from the tag libraries.  As it appears to me that is why tag
libraries are beans rather than replacements rules and code fragments in an
XML file.  But you could write some pretty fast taglibs with that repacement
rule file!

> Finally, it is conceivable that the implementation of these tags could
> possibly be reused by other JSP engines, or at least inspire development
of
> corresponding support in other publishing frameworks.  This could increase
> the ability of people to migrate their pages more easily from one
> technology or standard to another.  Whatever allegiance one might feel to
a
> particular specification - the level of openness is always goodness, and
is
> exactly what I have been working towards with PHP, Perl, and other
> integration efforts.

I could submit a patch that uses a deprecated 2.2 api that will allow jasper
to be used in any servlet 2.1 engine just by adding it as a servlet.  IMHO
we are basically there already.  I would just need to dig up some old notes
and muck around on the GnuJSP site so we could document how to do it.  In
fact to promote the use of tag libraries it would be a good thing.

>  - - - - - - - - - - - - - - - - - - - - - - - - -
>
> Part III - a plan to get there
>
> For starters, I would like to see tags originally introduced in order to
> support XML reintroduced into the code base of Jasper, for both tactical
> and strategic reasons as outlined above.  In parallel, the TLD for the
"JSP
> standard tag library" should be developed.  This would benefit greatly by
> people by contributions from those who have a deeply understanding of the
> spec.  In fact, I would argue that it would make more sense to incorporate
> the result into the subsequent revisions of the spec instead of merely
into
> Jakarta.

+1 on the "JSP Standard Actions Tag Library"  If I get time I can even write
it, I just need to get moved in an settled down.  Even though I don't think
they will replace the current handling of them because of performance
reasons, but if in the future, in say JSP2.0 there are some incompatible
changes made to the behavior of the tags then the TLD will be usefull to
authors who depended on the old behavior becuase they can breing the old
behavior in, just under a diffrent prefix.

0 on the JSP scriting elements as tag libraries.  Congrats if you can make
it work, I just am not into that sort of self inflicted pain.  If you figure
out how to get declerations reliably working let us all know.  The
<jsp:decleration/> tag has special handling written all over it.  Also
consider that they is no way to communicate out the value of the directives
to the engnie either, so the taglib would be a jasper only tag lib or would
need to require the parsing engine to implement some API to communicate the
values of those directives.  I just do not see how it can be done in under
one man year without implementing jasper itself inside of the tag library
with the special handling of non-action scripting elements in all their
glory.

> Once both are completed, Jasper's special case parsing logic should be
> changed to incorporate the new TLD and implement the majority of the
> standard tags in terms of more primitive tags, much in the way that the
> majority of the standard Java packages are themselves written in Java.  I
> believe that this can be done in a very incremental fashion.  Furthermore,
> the spec and the implementation should "co-evolve" for a period of time as
> some of the experience gained in implementing some of the stickier issues
> mentioned above may prove invaluable in helping form the correct
> specification for the desired behavior.
>
> Interestingly enough, if you look ahead to logical conclusion of this
> proposal, the likely conclusion is pretty much the opposite of the current
> situation: there will likely be a small kernel of built-in functions
> (including scriptlet), with the remainder moved out and into predefined
> extensions to the base.

I still think that there is value to having more that just scriptlets
handeled as a special case in the compiler.  Section 5.3.4 actually endorses
it for tag libraries.  Consider that for the typical JSP lifesycle the page
is compiled once and executed hundreds, thousands, maybe millions or more
times.  Now if for each page execution we are making the same decision each
time that invariably has the same result each time, such as which method
gets called to set the value of a particualr property, then those add up,
alot.  But if we could make that decision _once_, even if it takes ten times
as long to do it, and make the implemenation of those pages assume that
decision then almost always a time savings will be seen.  But one could make
some pathologincal example where this is not true, and my guess it will
invlove detecting an error condition earlier, but let's not optimize the
error condition.

Now I am not saying that Jasper is the best in this sense right now, but we
shouldn't be talking large steps in the other direction.  Implementing the
JSP standard actions as a tag library does have value, I just don't think
it's value is in replacing what is being done right now in the engine.

--Danno