Unable to parse an in-band nil on complex type

["Costello, Roger L." <co...@mitre.org>]

Hello DFDL community,

My input consists of a series of name, age pairs (on different lines):

John Doe
29
Sally Smith
34

This DFDL schema parses the input perfectly:

<xs:element name="input">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="person" maxOccurs="unbounded" dfdl:occursKind="implicit">
                <xs:complexType>
                    <xs:sequence dfdl:separator="%NL;" dfdl:separatorPosition="infix">
                        <xs:element name="name" type="xs:string" />
                        <xs:element name="age" type="xs:string" />
                    </xs:sequence>
                </xs:complexType>
            </xs:element>
        </xs:sequence>
    </xs:complexType>
</xs:element>


I want to test in-band nil on complex type. So, I added nillable="true" dfdl:nilValue="%ES;"
to the person element declaration:

<xs:element name="input">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="person" maxOccurs="unbounded" dfdl:occursKind="implicit" nillable="true" dfdl:nilValue="%ES;">
                <xs:complexType>
                   <xs:sequence dfdl:separator="%NL;" dfdl:separatorPosition="infix">
                        <xs:element name="name" type="xs:string" />
                        <xs:element name="age" type="xs:string" />
                    </xs:sequence>
                </xs:complexType>
            </xs:element>
        </xs:sequence>
    </xs:complexType>
</xs:element>

Now, when I parse the same input file, I get this error message:

Parse Error: Repeating or Optional Element - No forward progress at byte 29. Attempt to parse person succeeded but consumed no data.

Why am I getting this error? How do I fix it?

/Roger

Re: Unable to parse an in-band nil on complex type

["Beckerle, Mike" <mb...@tresys.com>]

The nil representation is preferred to any other representation, which is to say that the parser checks for the nil representation first before attempting to parse any other representation.

So if %ES; is the nil value, and there is no initiator nor terminator in the nil representation (based on dfdl:nilValueDelimiterPolicy) then DFDL will always create a nilled element and consume nothing.


If that happens in an array element where the array is of unbounded length, and if the array is not in a separated sequence, then you have created the DFDL equivalent of an infinite loop. You are able to positively parse and produce an element (in this case a nilled element), while not consuming any bits from the data stream at all, and you can do this an unbounded number of times. The DFDL spec. requires that implementations check for this case, and cause a parse error.


This check applies not specifically to nillable elements, but to any element representation. If zero bits are consumed, in an unbounded array context, then it's a parse error.


A requirement in DFDL is that you make forward progress through the data stream when parsing an unbounded array. You can consume nothing only in a scalar, or a bounded array, which insures that the array eventually ends even if nothing from the data stream is being consumed for each array element.


-mike beckerle


________________________________
From: Costello, Roger L. <co...@mitre.org>
Sent: Monday, July 15, 2019 9:13:39 AM
To: users@daffodil.apache.org
Subject: Unable to parse an in-band nil on complex type


Hello DFDL community,



My input consists of a series of name, age pairs (on different lines):



John Doe
29
Sally Smith
34



This DFDL schema parses the input perfectly:



<xs:element name="input">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="person" maxOccurs="unbounded" dfdl:occursKind="implicit">
                <xs:complexType>
                    <xs:sequence dfdl:separator="%NL;" dfdl:separatorPosition="infix">
                        <xs:element name="name" type="xs:string" />
                        <xs:element name="age" type="xs:string" />
                    </xs:sequence>
                </xs:complexType>
            </xs:element>
        </xs:sequence>
    </xs:complexType>
</xs:element>





I want to test in-band nil on complex type. So, I added nillable="true" dfdl:nilValue="%ES;"

to the person element declaration:



<xs:element name="input">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="person" maxOccurs="unbounded" dfdl:occursKind="implicit" nillable="true" dfdl:nilValue="%ES;">
                <xs:complexType>
                   <xs:sequence dfdl:separator="%NL;" dfdl:separatorPosition="infix">
                        <xs:element name="name" type="xs:string" />
                        <xs:element name="age" type="xs:string" />
                    </xs:sequence>
                </xs:complexType>
            </xs:element>
        </xs:sequence>
    </xs:complexType>
</xs:element>



Now, when I parse the same input file, I get this error message:



Parse Error: Repeating or Optional Element - No forward progress at byte 29. Attempt to parse person succeeded but consumed no data.



Why am I getting this error? How do I fix it?



/Roger