Bug in Daffodil? ... dynamically calculate the initiator and terminator -- neat!

["Costello, Roger L." <co...@mitre.org>]

Hello DFDL community,

My DFDL schema computes the initiator and terminator.

The input first has a number, followed by a comma-separated pair of values, possibly enclosed within round parentheses or square parentheses.

I ran my schema on this input:

1
(Apple,Banana)

And it generates this output:

<input>
  <num>1</num>
  <A>Apple</A>
  <B>Banana</B>
</input>

This input:

2
[Apple,Banana]

generates this output:

<input>
  <num>2</num>
  <A>Apple</A>
  <B>Banana</B>
</input>

But this input:

3
Apple,Banana

Throws the error shown below.

Here is my DFDL schema:

<xs:element name="input">
    <xs:complexType>
        <xs:sequence dfdl:separator="%NL;" dfdl:separatorPosition="infix">
            <xs:element name="num" type="xs:integer" />
            <xs:sequence dfdl:initiator="{if (num eq 1) then '(' else if (num eq 2) then '[' else ''}"
                dfdl:terminator="{if (num eq 1) then ')' else if (num eq 2) then ']' else ''}"
                dfdl:separator="," dfdl:separatorPosition="infix">
                <xs:element name="A" type="xs:string" />
                <xs:element name="B" type="xs:string" />
            </xs:sequence>
        </xs:sequence>
    </xs:complexType>
</xs:element>

Am I doing something wrong in calculating the initiator and terminator?  /Roger

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!   An unexpected exception occurred. This is a bug!   !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Please report this bug and help us fix it:

  https://daffodil.apache.org/community/#issue-tracker

Please include the following exception, the command you
ran, and any input, schema, or tdml files used that led
to this bug.


org.apache.daffodil.exceptions.Abort: Invariant broken: MPState.this.delimitersLocalIndexStack.length.==(1)
org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
org.apache.daffodil.processors.parsers.MPState.verifyFinalState(PState.scala:141)
org.apache.daffodil.processors.parsers.PState.verifyFinalState(PState.scala:345)
org.apache.daffodil.processors.DataProcessor.doParse(Runtime.scala:238)
        at org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
        at org.apache.daffodil.processors.parsers.MPState.verifyFinalState(PState.scala:141)
        at org.apache.daffodil.processors.parsers.PState.verifyFinalState(PState.scala:345)
        at org.apache.daffodil.processors.DataProcessor.doParse(Runtime.scala:238)
        at org.apache.daffodil.processors.DataProcessor.$anonfun$parse$1(Runtime.scala:196)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
        at org.apache.daffodil.ExecutionMode$.$anonfun$usingRuntimeMode$1(ExecutionMode.scala:65)
        at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:188)
        at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:184)
        at org.apache.daffodil.Main$.$anonfun$run$1(Main.scala:860)
        at org.apache.daffodil.util.Timer$.getTimeResult(Timer.scala:76)
        at org.apache.daffodil.util.Timer$.getResult(Timer.scala:35)
        at org.apache.daffodil.Main$.run(Main.scala:860)
        at org.apache.daffodil.Main$.main(Main.scala:1345)
        at org.apache.daffodil.Main.main(Main.scala)

Re: Bug in Daffodil? ... dynamically calculate the initiator and terminator -- neat!

[Steve Lawrence <sl...@apache.org>]

There's a couple problems here, some with the schema some with Daffodil.

The first is that your expressions for initiator and terminator can
return the empty string when the value of num is not 1 or 2. But the
DFDL expression for these properties is not allow the return an empty
string. We are probably throwing an SDE here to that affect, but looks
like it's being masked by some other bug in Daffodil. I'll open a ticket
for that.

The solution here is to return "%ES;" instead of the empty string. This
works for the initiator property, but since terminators are used for
calculating delimited lengths, you cannot use %ES; there. There
unfortunately is no way to to dynamically say that there is no
terminator. Instead, you'd need to do something like say the terminator
is "%NL;" and just require that the file end with a new line.

- Steve

On 4/24/19 1:49 PM, Costello, Roger L. wrote:
> Hello DFDL community,
> 
> My DFDL schema computes the initiator and terminator.
> 
> The input first has a number, followed by a comma-separated pair of values, 
> possibly enclosed within round parentheses or square parentheses.
> 
> I ran my schema on this input:
> 
> 1
> 
> (Apple,Banana)
> 
> And it generates this output:
> 
> <input>
> <num>1</num>
> <A>Apple</A>
> <B>Banana</B>
> </input>
> 
> This input:
> 
> 2
> 
> [Apple,Banana]
> 
> generates this output:
> 
> <input>
> <num>2</num>
> <A>Apple</A>
> <B>Banana</B>
> </input>
> 
> But this input:
> 
> 3
> 
> Apple,Banana
> 
> Throws the error shown below.
> 
> Here is my DFDL schema:
> 
> <xs:elementname="input">
> <xs:complexType>
> <xs:sequencedfdl:separator="%NL;"dfdl:separatorPosition="infix">
> <xs:elementname="num"type="xs:integer"/>
> <xs:sequencedfdl:initiator="{if (num eq 1) then '(' else if (num eq 2) then '[' 
> else ''}"
>                  dfdl:terminator="{if (num eq 1) then ')' else if (num eq 2) 
> then ']' else ''}"
>                  dfdl:separator=","dfdl:separatorPosition="infix">
> <xs:elementname="A"type="xs:string"/>
> <xs:elementname="B"type="xs:string"/>
> </xs:sequence>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> 
> Am I doing something wrong in calculating the initiator and terminator?  /Roger
> 
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> 
> !!   An unexpected exception occurred. This is a bug!   !!
> 
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> 
> Please report this bug and help us fix it:
> 
>    https://daffodil.apache.org/community/#issue-tracker
> 
> Please include the following exception, the command you
> 
> ran, and any input, schema, or tdml files used that led
> 
> to this bug.
> 
> org.apache.daffodil.exceptions.Abort: Invariant broken: 
> MPState.this.delimitersLocalIndexStack.length.==(1)
> 
> org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
> 
> org.apache.daffodil.processors.parsers.MPState.verifyFinalState(PState.scala:141)
> 
> org.apache.daffodil.processors.parsers.PState.verifyFinalState(PState.scala:345)
> 
> org.apache.daffodil.processors.DataProcessor.doParse(Runtime.scala:238)
> 
>          at org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
> 
>          at 
> org.apache.daffodil.processors.parsers.MPState.verifyFinalState(PState.scala:141)
> 
>          at 
> org.apache.daffodil.processors.parsers.PState.verifyFinalState(PState.scala:345)
> 
>          at org.apache.daffodil.processors.DataProcessor.doParse(Runtime.scala:238)
> 
>          at 
> org.apache.daffodil.processors.DataProcessor.$anonfun$parse$1(Runtime.scala:196)
> 
>          at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
> 
>          at 
> org.apache.daffodil.ExecutionMode$.$anonfun$usingRuntimeMode$1(ExecutionMode.scala:65)
> 
>          at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:188)
> 
>          at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:184)
> 
>          at org.apache.daffodil.Main$.$anonfun$run$1(Main.scala:860)
> 
>          at org.apache.daffodil.util.Timer$.getTimeResult(Timer.scala:76)
> 
>          at org.apache.daffodil.util.Timer$.getResult(Timer.scala:35)
> 
>          at org.apache.daffodil.Main$.run(Main.scala:860)
> 
>          at org.apache.daffodil.Main$.main(Main.scala:1345)
> 
>          at org.apache.daffodil.Main.main(Main.scala)
>