dfdl:emptyValueDelimiterPolicy doesn't do anything

["Costello, Roger L." <co...@mitre.org>]

The specification says this about dfdl:emptyValueDelimiterPolicy:

'none' indicates that if the content region is empty neither the dfdl:initiator or dfdl:terminator must be present. On unparsing when the content region is empty nothing will be output.

My DFDL schema (see below) declares two elements, separated by newline, each with an initiator, the first element specifies dfdl:emptyValueDelimiterPolicy="none". The input has empty content for the first element.

FIRST
SECOND Hello, world

Daffodil does not behave correctly: If I remove the initiator from the input file for the first element, I get an error. Bug in Daffodil?  /Roger

<xs:element name="input">
    <xs:complexType>
        <xs:sequence dfdl:separator="%NL;"
                               dfdl:separatorPosition="infix">
            <xs:element name="A" type="xs:string"
                                dfdl:initiator="FIRST"
                                dfdl:emptyValueDelimiterPolicy="none" />
            <xs:element name="B" type="xs:string"
                                 dfdl:initiator="SECOND" />
        </xs:sequence>
    </xs:complexType>
</xs:element>

Re: dfdl:emptyValueDelimiterPolicy doesn't do anything

["Beckerle, Mike" <mb...@tresys.com>]

This is one of the trickiest areas of DFDL. (ok, Maybe they're all tricky?)

It doesn't help that default values are only partly implemented in Daffodil.

In your example, the elements are required scalars, and have no default values, hence, emptyValueDelimiterPolicy is unused, as it makes no difference to anything. If the initiator is not found it will be an error.

dfdl:emptyValueDelimiterPolicy is intended to be used when a format requires a specific representation for an empty value. It applies only to required elements, and when an empty representation is found and if the element is defaultable, then the default value is substituted.

Consider

<xs:element name="x" type="xs:string" default="N/A"
      dfdl:initiator='[' dfdl:terminator=']'
      dfdl:emptyValueDelimiterPolicy="none"/
      minOccurs="3" maxOccurs="3"
      dfdl:occursCountKind="implicit">

So data like:

[A],,[C]

will result in

<x>A</x><x>N/A</x><x>C</x>

This unparses to:

[A][N/A][C]

If you change dfdl:emptyValueDelimiterPolicy="both", then to get that same infoset requires:

[A],[],[C]

and will unparse once again to:

[A][N/A][C]

However, daffodil only partly implements default values (only for parsing, and even then not all cases are covered I think), so I'm not sure the above will work.

Note if you unparse:

<x>A</x><x>C</x>

Then since there are 3 required elements, not 2, DFDL implementations should default a third value and you should get

[A],[C],[N/A]

I believe Daffodil will error here. It does not do defaulting on unparsing at all.


________________________________
From: Costello, Roger L. <co...@mitre.org>
Sent: Monday, September 16, 2019 10:40 AM
To: users@daffodil.apache.org <us...@daffodil.apache.org>
Subject: dfdl:emptyValueDelimiterPolicy doesn't do anything


The specification says this about dfdl:emptyValueDelimiterPolicy:



'none' indicates that if the content region is empty neither the dfdl:initiator or dfdl:terminator must be present. On unparsing when the content region is empty nothing will be output.



My DFDL schema (see below) declares two elements, separated by newline, each with an initiator, the first element specifies dfdl:emptyValueDelimiterPolicy="none". The input has empty content for the first element.



FIRST

SECOND Hello, world



Daffodil does not behave correctly: If I remove the initiator from the input file for the first element, I get an error. Bug in Daffodil?  /Roger



<xs:element name="input">
    <xs:complexType>
        <xs:sequence dfdl:separator="%NL;"
                               dfdl:separatorPosition="infix">
            <xs:element name="A" type="xs:string"
                                dfdl:initiator="FIRST"
                                dfdl:emptyValueDelimiterPolicy="none" />
            <xs:element name="B" type="xs:string"
                                 dfdl:initiator="SECOND" />
        </xs:sequence>
    </xs:complexType>
</xs:element>