Why am I getting unconsumed bytes in this choice within a choice?

["Costello, Roger L." <co...@mitre.org>]

Hi Folks,

I have a binary file. It has a choice of 2-byte "markers". The last 2 bytes in the file must be FF D9.

One of the markers is FF E1. To see if my approach works, I created a choice for marker FF E1 and nothing else. I placed that choice within another choice which has an element that gobbles up all the remaining bytes except for the last 2 bytes. Here is my choice within a choice:

<xs:choice>
    <xs:choice>
            <xs:element name="APP1" type="xs:hexBinary" 
                dfdl:lengthKind="pattern"
                dfdl:lengthPattern="\xFF\xE1"
                dfdl:lengthUnits="bytes"
            />
    </xs:choice>
    <xs:element name="Bytes" type="xs:hexBinary" 
        dfdl:lengthKind="pattern"
        dfdl:lengthPattern="[\x00-\xFF]+?(?=\xFF\xD9$)"
        dfdl:lengthUnits="bytes"
    />
</xs:choice>

When I run it, Daffodil reports "Left over data." Daffodil consumed FF E1 and then stopped. Any thoughts on what I'm doing wrong?

/Roger

Re: Why am I getting unconsumed bytes in this choice within a choice?

["Beckerle, Mike" <mb...@tresys.com>]

I don't think you want nested choices here.

When you have nested choices, only one of the "leaves" of the nest of choices will ever be taken. So your nest here must resolve to either your APP1 element or your Bytes element, but never one then the other.

Your APP1 element if matched, is the only thing, and just consumes those two bytes. Anything after the APP1 element in the data stream is going to be left over data.

________________________________
From: Costello, Roger L. <co...@mitre.org>
Sent: Monday, May 11, 2020 12:52 PM
To: users@daffodil.apache.org <us...@daffodil.apache.org>
Subject: Why am I getting unconsumed bytes in this choice within a choice?

Hi Folks,

I have a binary file. It has a choice of 2-byte "markers". The last 2 bytes in the file must be FF D9.

One of the markers is FF E1. To see if my approach works, I created a choice for marker FF E1 and nothing else. I placed that choice within another choice which has an element that gobbles up all the remaining bytes except for the last 2 bytes. Here is my choice within a choice:

<xs:choice>
    <xs:choice>
            <xs:element name="APP1" type="xs:hexBinary"
                dfdl:lengthKind="pattern"
                dfdl:lengthPattern="\xFF\xE1"
                dfdl:lengthUnits="bytes"
            />
    </xs:choice>
    <xs:element name="Bytes" type="xs:hexBinary"
        dfdl:lengthKind="pattern"
        dfdl:lengthPattern="[\x00-\xFF]+?(?=\xFF\xD9$)"
        dfdl:lengthUnits="bytes"
    />
</xs:choice>

When I run it, Daffodil reports "Left over data." Daffodil consumed FF E1 and then stopped. Any thoughts on what I'm doing wrong?

/Roger