You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@daffodil.apache.org by Roger L Costello <co...@mitre.org> on 2022/03/25 11:46:37 UTC
Recommended changes to the DFDL language
Hi Folks,
I am converting 350 military data formats to DFDL.
Everyone of the 350 data formats has an optional choice:
<xs:choice minOccurs="0">
DFDL does not allow optional choice. Why?
I recommend changing the DFDL language to allow optional choice.
Everyone of the 350 data formats has branches of a choice that are optional:
<xs:choice>
<xs:element name="foo" minOccurs="0">
DFDL does not allow branches in a choice to be optional. Why?
I recommend changing the DFDL language to allow optional branches of a choice.
Many of the 350 data formats have an element with complexType that is nillable and the nilValue is a hyphen.
DFDL does not allow an element with complexType to have a nilValue other than %ES;
I recommend changing the DFDL language to allow elements with complexType to have a nilValue other than %ES;
/Roger
RE: Recommended changes to the DFDL language
Posted by "Brutzman, Donald (Don) (CIV)" <br...@nps.edu>.
Concur that optional choices and also nillable are fundamentally important
data-representation capabilities to support.
Note that similar issues occur for attributes, which also need to be
supported by DFDL. When we round-trip convert element+attribute XML to
element-only XML so that DFDL can handle it, similar data-representation
issues can emerge.
p.s. As with skateboards - XML attributes are not a crime. 8)
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman@nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, Navy robotics https://
faculty.nps.edu/brutzman
-----Original Message-----
From: Roger L Costello <co...@mitre.org>
Sent: Friday, March 25, 2022 4:47 AM
To: users@daffodil.apache.org
Subject: Recommended changes to the DFDL language
NPS WARNING: *external sender* verify before acting.
Hi Folks,
I am converting 350 military data formats to DFDL.
Everyone of the 350 data formats has an optional choice:
<xs:choice minOccurs="0">
DFDL does not allow optional choice. Why?
I recommend changing the DFDL language to allow optional choice.
Everyone of the 350 data formats has branches of a choice that are optional:
<xs:choice>
<xs:element name="foo" minOccurs="0">
DFDL does not allow branches in a choice to be optional. Why?
I recommend changing the DFDL language to allow optional branches of a
choice.
Many of the 350 data formats have an element with complexType that is
nillable and the nilValue is a hyphen.
DFDL does not allow an element with complexType to have a nilValue other
than %ES;
I recommend changing the DFDL language to allow elements with complexType to
have a nilValue other than %ES;
/Roger
Re: Recommended changes to the DFDL language
Posted by Mike Beckerle <mb...@apache.org>.
Well, the good news is that your 3rd item, about allowing hyphens (or non
ES generally) as nilValue for complex type elements is targeted for release
3.4.0 of daffodil as an experimental extension to DFDL which assuming it
works out, we would propose for inclusion in a future DFDL revision. (Now
we just have to find someone to work on it!)
It is ticket https://issues.apache.org/jira/browse/DAFFODIL-2636
As for minOccurs 0 on model groups. 3 arguments against it.
1) added complexity
2) Too XML centric/specific
3) Technical issues associated with multiple points of uncertainty (for
choices)
So (1) Sequences can't have minOccurs="0" for simply the reason that DFDL
is complex already. Adding array/optional behavior to anonymous model
groups is very problematic.
One example area is expressions. Allowing repeating or optional anonymous
groups complicates path expressions immensely. DFDL indexing is dead simple
in comparison to XPath "indexing", which can't really be compiled easily.
XPath indexing is much more like a query than like array element access.
They don't even call it indexing, they call it "predicates". DFDL path
expressions are always executable in constant time, or that was the
intention anyway. Each step of a path expression is either up, down to a
named scalar child element, or down to a single specific index in a named
array/optional. All steps can be compiled to tiny non-complex operations.
By comparison, XPath indexing is O(n) in the size of the data, and can
return a node list that is O(n) in size.
Re: (2) Note that anonymous things that are arrays or optionals are not a
feature of any structured data format notation I know of, *except* XML.
There are numerous other XML-isms that were rejected in DFDL e.g., no
attribute children of elements. Again this is because there is no
structured data format notation that allows two different kinds of children
with separate namespaces except XML.
re: (3) Choices can't have minOccurs="0" because then there would be two
points of uncertainty in one location. Does a dfdl:discriminator
discriminate both, or just the choice, or just the optionality?
Now, DFDL is an "open" standard, and if people want to participate in the
DFDL workgroup directly, and evolve the standard to better accomodate the
needs of XML users, then that's certainly one direction it could go in.
Apache Daffodil is an open-source implementation suitable for prototyping
and it goes in the direction its developers take it.
We acknowledge the fact that XML users find DFDL restrictive compared with
XSD, but we balance that against the fact that numerous people find DFDL
far too XML oriented and complex. This is reflected in our Apache Daffodil
Roadmap (
https://cwiki.apache.org/confluence/display/DAFFODIL/Roadmap+for+Upcoming+Releases).
Software developers definitely have a love or hate relationship with XML
and there is little middle ground. See specifically sections of the Roadmap
titled: "XML Feature Enhancements (and JSON)" and "Escape from XML - A
Non-XSD Syntax for DFDL".
On Fri, Mar 25, 2022 at 7:46 AM Roger L Costello <co...@mitre.org> wrote:
> Hi Folks,
>
> I am converting 350 military data formats to DFDL.
>
> Everyone of the 350 data formats has an optional choice:
>
> <xs:choice minOccurs="0">
>
> DFDL does not allow optional choice. Why?
>
> I recommend changing the DFDL language to allow optional choice.
>
> Everyone of the 350 data formats has branches of a choice that are
> optional:
>
> <xs:choice>
> <xs:element name="foo" minOccurs="0">
>
> DFDL does not allow branches in a choice to be optional. Why?
>
> I recommend changing the DFDL language to allow optional branches of a
> choice.
>
> Many of the 350 data formats have an element with complexType that is
> nillable and the nilValue is a hyphen.
>
> DFDL does not allow an element with complexType to have a nilValue other
> than %ES;
>
> I recommend changing the DFDL language to allow elements with complexType
> to have a nilValue other than %ES;
>
> /Roger
>
Re: Recommended changes to the DFDL language
Posted by Steve Lawrence <sl...@apache.org>.
One alternative for minOccurs on a choice is to have an empty branch:
<xs:choice>
<xs:element name="foo" ... />
<xs:sequence />
</xs:choice>
I admit that minOccurs="0" is a little more self documenting.
I think minOccurs also means something different if there are delimiters
on the choice. For example, say we had these two snippets to implement
an optional choice with a terminator:
<xs:choice dfdl:terminator="X">
<xs:element name="foo" ... />
<xs:sequence />
</xs:choice>
and
<xs:choice dfdl:terminator="X" minOccurs="0">
<xs:element name="foo" ... />
</xs:choice>
In the former, if the X terminator doesn't exist, then it is a parse
error. But in the later case with minOccurs, if the X terminator doesn't
exist then it just means the choice doesn't exist.
So the semantics are a bit different. All that said, I don't think I've
come across a case where we've absolutely needed minOccurs="0" on a
choice, and not allowing it keeps DFDL simpler.
As to nillable complex other than %ES;, we have a ticket open to add
this as an experimental feature, currently slated for the next release:
https://issues.apache.org/jira/browse/DAFFODIL-2636
On 3/25/22 7:46 AM, Roger L Costello wrote:
> Hi Folks,
>
> I am converting 350 military data formats to DFDL.
>
> Everyone of the 350 data formats has an optional choice:
>
> <xs:choice minOccurs="0">
>
> DFDL does not allow optional choice. Why?
>
> I recommend changing the DFDL language to allow optional choice.
>
> Everyone of the 350 data formats has branches of a choice that are optional:
>
> <xs:choice>
> <xs:element name="foo" minOccurs="0">
>
> DFDL does not allow branches in a choice to be optional. Why?
>
> I recommend changing the DFDL language to allow optional branches of a choice.
>
> Many of the 350 data formats have an element with complexType that is nillable and the nilValue is a hyphen.
>
> DFDL does not allow an element with complexType to have a nilValue other than %ES;
>
> I recommend changing the DFDL language to allow elements with complexType to have a nilValue other than %ES;
>
> /Roger