HEADS UP - cocoon form handling (long!!)

[Torsten Curdt <tc...@dff.st>]

HEADS UP - cocoon form handling
-------------------------------

There has been quite a long dicussion off the list about the new cocoon
form handling stuff. In order to get more opinions we would like to 
present an excerpt of the current results and thoughts to a wider 
audience and ask for comments.

Ivelin and I tried to come up with a more general approach that can
combine both approaches that are currently in the scratchpad (xmlform 
and precept). While xmlform is somehow tied to schematron validation, 
precept tries to provide a framework to plug in different validation 
schemes. We aggreed on the final goal of a general form validation 
framework that allows different validation schemes (which includes 
schematron as well as relaxng and XML Schema).
We'd like everyone interested on this subject to try out the examples 
provided for xmlform and precept in the scratchpad.

http://cvs.apache.org/viewcvs.cgi/xml-cocoon2/src/scratchpad/webapp/mount/precept/
http://cvs.apache.org/viewcvs.cgi/xml-cocoon2/src/scratchpad/webapp/mount/xmlform/

and join the discussion on cocoon-dev.

                       INTRODUCTION OF TERMS

                               -o-
                           The Instance

Form handling is always about collecting and validating user input that
finally gets submitted. Since this collection process is not always
finished in a single request-cycle (wizards) you need some kind of store
to save the data until you can finally submit it. This store can be 
either a DOM or Beans. We call the general store of those data 
"instance" according to the current XForms working draft.

                               -o-
                              Views

This instance needs to be filled by the submission of one or more pages 
of the current media. (we will focus on HTML right now) So what the user
actually sees might only be a "view" or a part of the instance. And only
this part will be POSTed and saved into the instance when the user hits
the "next" button.

       [------------instance-----------]
HTML: [-----view1-----][-----view2----]
WML:  [--view1--][--view2--][--view3--]

The view maps values from the instance to controls. Syntax reminds on
XForms:

   <i:instance id="feedback-form">
     <i:textbox ref="user/firstname"/>
     <i:textbox ref="user/lastname"/>
     ...
   </i:instance>

                               -o-
                             Populating

Taken from the ExFormular project we call this saving into the instance 
on a POST "populating". There are two different approaches when 
populating. Direct and indirect. While the direct populating method is 
quite straight forward on the first glance: "go through the request and 
save the request parameters into the instance" it has some major 
problems as of the POST behaviour of checkboxes: on "true" there will be 
a request parameter, on "false" there will be no request parameter at 
all. So with direct population you cannot set checkboxes to "false". The 
other method is indirect population. With this approach you actually 
*know* what parameters *should* come with the request. So a missing 
checkbox parameter means "set this checkbox to false". Although we could 
use the direct population (with some workarounds) we aggreed on using 
indirect populating as it is the cleaner approach in the long run. No 
hidden field or something like that...

                               -o-
                             Phases

A phase defines a set of nodes of an instance. Phases are usally used
in the validation and population process.

                               -o-
                            Preceptor

Our instance can be expressed in XML. Directly when it's a DOM or mapped
via Castor when we have beans. This means we should be able to use the
commonly used validation methods to validate our instance (= data of a
form). We would usually  call them Schemas but since the term
collides with short form of the W3C XML Schema validation we aggreed on
"preceptor" for a general validity description of an XML document. Such 
a desciption will be served via pipeline and then "compiled" and saved 
into the scope of the instance. The JARV project shares almost the same 
idea here. Unfortunately they do not have support for partial validation 
yet, too.

There are two different parts of XML document validity:
  a) structure of the document
  b) content of nodes
                               -o-
                          Content Validation

Let's focus on the content first. It's the most important thing for form
validation. Each preceptor defines some kind of rules, restrictions,
assertations for the content of nodes of the instance. As general term 
we will call them "constraints". Probably the most important constraints 
from the grammar based world (schema,relaxng) are

  *) regular expressions constraint
  *) enumeration or choice constraint
  *) minimum number value constraint
  *) maxmimum number value constraint
  *) minimum string length constraint
  *) maximum string length constraint
  *) ...

While schematron only has a single but very powerful constraint that
combines all the above into the

  *) XPath expression constraint (except for regex until XPath 2.0)

                               -o-
                          Structure Validation

The structure validation takes care of the instance to conform to a
described xml structure. We have not yet aggreed if the structure should
be fully described by the preceptor or a partial describtion may be
sufficient. This decision has a major impact on the building of the
instance and the implementation details.

                               -o-
                          Building the Instance

At the start of the flow an instance needs to be created to take the
values of the POSTs. If we want to support structure validation we need 
to make sure that already the empty instance conforms to the preceptor. 
As of this we need to create all the required nodes in this instance then.

                       A SIMPLE FORM VALIDATION

This is how we would like to see form validation in action:

  1. Introspection happens at the start of flow. An empty instance gets
     created and save into the desired scope. It can also get filled
     with some inital values.

  2. The controller (an action or schecoon) will choose the first view to
     present to the user.

  3. If the user now presses a (submit) button the values get populated
     into the instance and then validated. If some constraints failed we
     go back to the first view telling the user which constraint have
     failed otherwise present him the next view.

Unfortunately this looks much easier at the first glance than it is. We
will now try to talk about the problems we have encounter so far.
(and there possible sollutions ;)

on 1)
   How do we specify an introspection for a flow?

     This can easily be done either with schecoon or with the multiaction
     that comes with precept. But the question is: Should this be modeled
     in XML, in java or a scripting language?

on 2)
   How does the controller know which view is the first one and what
   views are actually available?

     This could be set from the introspection. If not specified the first
     view from a possible descriptor could be taken.
     See also "VIEWS AND PHASES".

on 3)
   How does the controller know what to populate and validate on a
   submit?

     Inside the views or in a descriptor like it is in Struts we should
     probably have a mapping of the phases to buttons.
     See "VIEWS AND PHASES".

   And how does he know about the current and the next view?

     Same as above. This can happen in the controler or be specified
     in the view.
     See "VIEWS AND PHASES".

   What if we don't want to populate and validate all controls in a view?

     We need to keep in mind to decouple the phases from the views
     See "VIEWS AND PHASES".

   How do we present the failed constraints to the user?

      We can use the InstanceTransformer to insert a list/tree of errors
      into the SAX stream somewhere in the view or output the failed
      constraints with the controls itself.

   How can we only validate parts of an instance?

      Unfortunately most current validation implementations don't
      support partial validation of documents yet. Schematron is
      one exception. Maybe some of the authors can assist here.

Since much of the problems are phases / views related we took a closer look.


                       VIEWS AND PHASES


First thing our discussion revealed was that views and phases are not
necessarily the same. They might be the same for the most easy webforms
but as soon as it comes to more complex applications you need to 
decouple them. They describe different things:

  *) phases describe a set of instance nodes to be validated
  *) views describe a set of instance nodes to be displayed

So as a result we found that buttons should be bound to phases.
Either within the view or within a descriptor:

a) within the view
<view>
   <instance id="..">
     <textbox ref="user/firstname"/>
     <textbox ref="user/lastname"/>
     <button>
       <caption>Next</caption>
       <populate>phase2</populate>
       <validate>phase2</validate>
     </button>
   </instance>
</view>

b) within a descriptor

    <no syntax yet/>

Currently only schematron comes with a native phases support. All the
others don't support partial validation yet. Although Torsten is not so
sure if this should be really part of the preceptor.
(Torsten: as soon as you move a textbox from one view into another one 
you need alter your preceptor because the phases have changed. For SoC 
this should not be IMHO)


But the question still remains: how can we implement phases for 
preceptor other than schematron? All we came up was some kind of 
PhaseResolver than can takes the same schematron file as input for the 
SchemtronPhaseResolver and other descriptors (to be defined) for e.g. 
the RelaxNGPhaseResolver.

                       IMPLEMENTATION DETAILS

 From the above we came up with some minimalistic interfaces:

/**  the instance save and retrieves values based on an xpath
   */
interface Instance {
   public void setValue( String xpath, Object value, Context context);
   public Object getValue( String xpath );
}

/**  the preceptor validates nodes from a specific phase looked up via 
the phase resolver
   *
   */
interface Preceptor {
   public Collection validate( Instance i, PhaseResolver resolver, 
String phase, Context context );
   /* or better typesafe?
   public Constraint[] validate( Instance i, PhaseResolver resolver, 
String phase, Context context );
   */
}

/**  the constraint actually checks the the values of a node.
   *  it's usually a wrapper to existing "restrictions,rules,..."
   */
interface Constraint {
   public boolean isSatifiedBy( Object value, Context context );
   public String getId();
}

/**  the phase resolver returns the xpaths for for phase
   */
interface PhaseResolver {
   public String[] lookupNodesForPhase( String phase );
}


                         COMMENTS

from Torsten:
- I am in favor of "structure validation" of forms. So all I define is 
the preceptor (xsd or relaxng) and this even describes how the instance 
has to look like. And I am then sure my instance looks like the preceptor.
- I am in favor of grammar markups like xsd or relaxng because then 
constraints (e.g. EnumerationConstraints) can easily be used to fill 
e.g. comboboxes (see example2 in precept) with e.g. schematron all this 
is "hidden" in expressions
- Later I like to add an interface ClientJavaScript (or somthing like 
that) so that constraints can produce javascript code for client validation

from Ivelin:
<nothing yet/>

Puh! Now it's your time ;)
Any comments are welcome!!!
--
Torsten + Ivelin


---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org

Re: HEADS UP - cocoon form handling (long!!)

[Stefano Mazzocchi <st...@apache.org>]

Torsten Curdt wrote:
> 
> HEADS UP - cocoon form handling
> -------------------------------
> 
> There has been quite a long dicussion off the list about the new cocoon
> form handling stuff. In order to get more opinions we would like to
> present an excerpt of the current results and thoughts to a wider
> audience and ask for comments.

Just one thought: I'm very impressed by the quality of the work and of
the architectural thinking that you guys are placing into this feature
that I perfectly agree will make Cocoon much more useful and reduce some
of the 'asymmetry' that Cocoon exhibits and it's probably due to my lack
of personal experience in this realm.

So, if you don't hear from me on this it's not because I don't like what
you are doing but rather the opposite: I keep silent when I don't have
anything smart to say :)

[doesn't mean that when I say something it's always smart, no, but at
least I *think* I can contribute something.... here, I'm not sure]

-- 
Stefano Mazzocchi      One must still have chaos in oneself to be
                          able to give birth to a dancing star.
<st...@apache.org>                             Friedrich Nietzsche
--------------------------------------------------------------------



---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org