You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by Christofer Dutz <ch...@c-ware.de> on 2019/01/15 15:39:03 UTC
Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
Hi all,
after working though the 6 tutorials on DFDL in general I think I have a much greater understanding on how I have to do things. I even managed to get my S7 protocol messages schema in a somewhat working condition.
Right now I’m having one problem:
A S7 Messages consists of a header, a number of variable length parameters and a number of variable length payloads.
As parameters and payloads are of variable length, the header contains a “parametersLength” and “payloadsLength” field which contains the number of bytes the parameters and payloads require in total.
So I defined something like this:
<xs:element name="parametersLength" type="s7:short"/>
<xs:element name="payloadsLength" type="s7:short"/>
<xs:element name="parameters" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}">
<xs:complexType>
<xs:sequence>
<xs:choice>
<xs:element ref="s7:S7GeneralParameterSetupCommunication"/>
<xs:element ref="s7:SS7RequestParameterCPUService"/>
<xs:element ref="s7:S7RequestParameterReadVar"/>
<xs:element ref="s7:S7RequestParameterWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="payloads" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}">
<xs:complexType>
<xs:sequence>
<xs:choice>
<xs:element ref="s7:S7RequestPayloadCpuServices"/>
<xs:element ref="s7:S7RequestPayloadWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
However in a request, that doesn’t contain any payloads, daffodil still tries to parse the payloads, even if the “length” of the sequence is set to explicit and to a length of 0 … why is it doing that?
Each parameter and payloads first byte contains the code that tells the parser what type it is and each of the elements in my schema use a discriminator to tell the parser which input it is requiring.
<xs:element name="S7RequestPayloadCpuServices">
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:discriminator test="{./type eq 0}"/>
</xs:appinfo>
</xs:annotation>
<xs:complexType>
<xs:sequence>
<xs:element name="type" type="s7:byte"/>
<xs:element name="transportSize" type="s7:byte"/><!-- fixed="9"-->
<xs:element name="length" type="s7:byte"/>
<xs:element name="sslId" type="s7:short"/>
<xs:element name="sslIndex" type="s7:short"/>
</xs:sequence>
</xs:complexType>
</xs:element>
Hope it’s correct to do things like that …
The effect is that Daffodil has correctly parsed all the parameters and as there is no payload (payloadLength = 0) it shouldn’t try to parse a payload, but it does and when reading the first byte it instantly fails as there is no data to parse anymore.
Chris
Re: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
Posted by "Beckerle, Mike" <mb...@tresys.com>.
Christofer,
So looking at your schemas I think the concept you are missing is that just because something is zero length doesn't mean the parser won't try to parse against it.
A DFDL schema could parse, with zero data, consume zero bits of course, yet still populate an infoset with default values, empty strings, zero-length hexBinary byte arrays, nilled elements, or computed elements. The parser doesn't turn on/off based on zero length being available.
What you need do is use minOccurs="0", and dfdl:occursCountKind="expression" and dfdl:occursCount="{ if (../parametersLength gt 0) then 1 else 0 }"
What is maybe a little confusing is that you actually must specify both - the length AND whether there are occurrences or not. If there are no occurrences, the length won't be used, but if there are, the parser must be able to determine how long they are.
...mike beckerle
Tresys
________________________________
From: Christofer Dutz <ch...@c-ware.de>
Sent: Tuesday, January 15, 2019 10:39:03 AM
To: dev@daffodil.apache.org
Subject: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
Hi all,
after working though the 6 tutorials on DFDL in general I think I have a much greater understanding on how I have to do things. I even managed to get my S7 protocol messages schema in a somewhat working condition.
Right now I’m having one problem:
A S7 Messages consists of a header, a number of variable length parameters and a number of variable length payloads.
As parameters and payloads are of variable length, the header contains a “parametersLength” and “payloadsLength” field which contains the number of bytes the parameters and payloads require in total.
So I defined something like this:
<xs:element name="parametersLength" type="s7:short"/>
<xs:element name="payloadsLength" type="s7:short"/>
<xs:element name="parameters" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}">
<xs:complexType>
<xs:sequence>
<xs:choice>
<xs:element ref="s7:S7GeneralParameterSetupCommunication"/>
<xs:element ref="s7:SS7RequestParameterCPUService"/>
<xs:element ref="s7:S7RequestParameterReadVar"/>
<xs:element ref="s7:S7RequestParameterWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="payloads" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}">
<xs:complexType>
<xs:sequence>
<xs:choice>
<xs:element ref="s7:S7RequestPayloadCpuServices"/>
<xs:element ref="s7:S7RequestPayloadWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
However in a request, that doesn’t contain any payloads, daffodil still tries to parse the payloads, even if the “length” of the sequence is set to explicit and to a length of 0 … why is it doing that?
Each parameter and payloads first byte contains the code that tells the parser what type it is and each of the elements in my schema use a discriminator to tell the parser which input it is requiring.
<xs:element name="S7RequestPayloadCpuServices">
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:discriminator test="{./type eq 0}"/>
</xs:appinfo>
</xs:annotation>
<xs:complexType>
<xs:sequence>
<xs:element name="type" type="s7:byte"/>
<xs:element name="transportSize" type="s7:byte"/><!-- fixed="9"-->
<xs:element name="length" type="s7:byte"/>
<xs:element name="sslId" type="s7:short"/>
<xs:element name="sslIndex" type="s7:short"/>
</xs:sequence>
</xs:complexType>
</xs:element>
Hope it’s correct to do things like that …
The effect is that Daffodil has correctly parsed all the parameters and as there is no payload (payloadLength = 0) it shouldn’t try to parse a payload, but it does and when reading the first byte it instantly fails as there is no data to parse anymore.
Chris
Re: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
Posted by Steve Lawrence <sl...@apache.org>.
This looks really good to me and should have pretty good performance.
I would make one minor suggestion, which is to change your payload
choiceDispatchKey expression from:
{../../parameters[1]/parameter[fn:count(../payload)]/type}
to
{../../parameters[1]/parameter[dfdl:occursIndex()]/type}
So use the dfdl:occursIndex() function to find the associated parameter
instead of the fn:count() function. The dfdl:occursIndex() function
returns the index of the current array, which in this context is the
payload array. This is functionally the same as fn:count(../payload),
but dfdl:occursIndex() should be a bit more efficient. This is because
fn:count(../payload) needs to evaluate the ../payload path to find the
payload array and then count the number of elements, whereas
dfdl:occursIndex() can access internal state to know the exact current
index in constant time.
- Steve
On 1/16/19 7:28 AM, Christofer Dutz wrote:
> Hi all,
>
> I think I found a solution on my own ... the key is that I now output an payload element for every parameter, even if that is empty in some cases, with this, I was able to do the following:
>
> <xs:element name="S7ResponseMessage">
> <xs:complexType>
> <xs:sequence>
> <!-- Reserved value always 0x0000 -->
> <xs:element name="reserved" type="s7:short" fixed="0"/>
> <xs:element name="tpduReference" type="s7:short"/>
> <xs:element name="parametersLength" type="s7:short"/>
> <xs:element name="payloadsLength" type="s7:short"/>
> <!-- UserData (type 7) responses don't have the error class and code -->
> <xs:element name="errorClass" type="s7:byte" minOccurs="0"
> dfdl:occursCountKind="expression" dfdl:occursCount="{if(../../type eq 3) then 1 else 0}"/>
> <xs:element name="errorCode" type="s7:byte" minOccurs="0"
> dfdl:occursCountKind="expression" dfdl:occursCount="{if(../../type eq 3) then 1 else 0}"/>
> <xs:element name="parameters" minOccurs="0"
> dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}"
> dfdl:occursCountKind="expression" dfdl:occursCount="{if(../parametersLength gt 0) then 1 else 0}">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="parameter" maxOccurs="unbounded">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="type" type="s7:byte"/>
> <xs:choice dfdl:choiceDispatchKey="{type}">
> <xs:element dfdl:choiceBranchKey="240" ref="s7:S7GeneralParameterSetupCommunication"/>
> <xs:element dfdl:choiceBranchKey="0" ref="s7:S7ResponseParameterCPUService"/>
> <xs:element dfdl:choiceBranchKey="4" ref="s7:S7ResponseParameterReadVar"/>
> <xs:element dfdl:choiceBranchKey="5" ref="s7:S7ResponseParameterWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> <xs:element name="payloads" minOccurs="0"
> dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}"
> dfdl:occursCountKind="expression" dfdl:occursCount="{if((../payloadsLength gt 0) or (../parametersLength gt 0)) then 1 else 0}">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="payload" maxOccurs="unbounded"
> dfdl:occursCountKind="expression" dfdl:occursCount="{fn:count(../../parameters[1]/parameter)}">
> <xs:complexType>
> <xs:sequence>
> <xs:choice dfdl:choiceDispatchKey="{../../parameters[1]/parameter[fn:count(../payload)]/type}">
> <xs:element dfdl:choiceBranchKey="240" ref="s7:S7GeneralPayloadSetupCommunication"/>
> <xs:element dfdl:choiceBranchKey="0" ref="s7:S7ResponsePayloadCpuServices"/>
> <xs:element dfdl:choiceBranchKey="4" ref="s7:S7ResponsePayloadReadVar"/>
> <xs:element dfdl:choiceBranchKey="5" ref="s7:S7ResponsePayloadWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> So what I do now is that I use the type-element of the matching parameter for the second decision.
>
> However I would like to report, das not even by debugging the code was I able to understand the error message about not supporting arrays.
> "Query-style paths not supported. Must have '[...]' after array-element's name."
> Cause the real reason is that the problem was that on the up-path only the last part can exist without the array notation and in this case the intermediate "parameters" fragment had to be added an array index.
>
> How is the above solution performance-wise?
>
> Chris
>
>
>
> Am 16.01.19, 11:54 schrieb "Christofer Dutz" <ch...@c-ware.de>:
>
> Hi all,
>
> thanks for all of your help ... I am coming closer and closer to a first fully operational DFDL schema for S7 communication ... really looking forward to experimenting with that.
>
> Now I ran into something probably quite special:
>
> In my messages I have a header, parameters and payloads
> While each payload has a "type" indicator, the payloads don't. The payloads are somewhat parsed by iterating over the previously parsed parameters and for each parameter (in exactly the same order) create a payload.
> However not all parameters have payloads ...
>
> So here's what I've come up so far:
>
> <xs:element name="S7ResponseMessage">
> <xs:complexType>
> <xs:sequence>
> <!-- Reserved value always 0x0000 -->
> <xs:element name="reserved" type="s7:short" fixed="0"/>
> <xs:element name="tpduReference" type="s7:short"/>
> <xs:element name="parametersLength" type="s7:short"/>
> <xs:element name="payloadsLength" type="s7:short"/>
> <!-- UserData (type 7) responses don't have the error class and code -->
> <xs:element name="errorClass" type="s7:byte" minOccurs="0"
> dfdl:occursCountKind="expression" dfdl:occursCount="{if(../../type eq 3) then 1 else 0}"/>
> <xs:element name="errorCode" type="s7:byte" minOccurs="0"
> dfdl:occursCountKind="expression" dfdl:occursCount="{if(../../type eq 3) then 1 else 0}"/>
> <xs:element name="parameters" minOccurs="0"
> dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}"
> dfdl:occursCountKind="expression" dfdl:occursCount="{if(../parametersLength gt 0) then 1 else 0}">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="parameter" maxOccurs="unbounded">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="type" type="s7:byte"/>
> <xs:choice dfdl:choiceDispatchKey="{type}">
> <xs:element dfdl:choiceBranchKey="240" ref="s7:S7GeneralParameterSetupCommunication"/>
> <xs:element dfdl:choiceBranchKey="0" ref="s7:S7ResponseParameterCPUService"/>
> <xs:element dfdl:choiceBranchKey="4" ref="s7:S7ResponseParameterReadVar"/>
> <xs:element dfdl:choiceBranchKey="5" ref="s7:S7ResponseParameterWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> <xs:element name="payloads" minOccurs="0"
> dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}"
> dfdl:occursCountKind="expression" dfdl:occursCount="{if(../payloadsLength gt 0) then 1 else 0}">
> <xs:complexType>
> <xs:sequence>
> <xs:element name="payload" maxOccurs="unbounded"
> dfdl:occursCountKind="expression" dfdl:occursCount="{count(../parameters/parameter)}">
> <xs:complexType>
> <!-- TODO: Somehow loop over the parameters and use those elements as type keys -->
> <xs:sequence>
> <xs:element name="type" type="s7:byte"/>
> <xs:choice dfdl:choiceDispatchKey="{type}">
> <xs:element dfdl:choiceBranchKey="0" ref="s7:S7ResponsePayloadCpuServices"/>
> <xs:element dfdl:choiceBranchKey="4" ref="s7:S7ResponsePayloadReadVar"/>
> <xs:element dfdl:choiceBranchKey="5" ref="s7:S7ResponsePayloadWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> So the "payload" element should occur exactly as often as there are parameters. Now I was thinking that if for a parameter there is no matching payload, I'll just not output any or just some dummy output.
> The problem however is, how can I iterate over the parameters and for each get the "type" and depending on that select the right choice branch?
>
> Chris
>
>
> Am 16.01.19, 02:08 schrieb "Beckerle, Mike" <mb...@tresys.com>:
>
> Christofer,
>
>
> So in DFDL, unlike regular XSD, only elements can be repeating/optional.
>
>
> You can't have max/minOccurs on a choice or sequence, only on an element.
>
>
> DFDL isn't only for XML. Many data models that the DFDL infoset could be projected into, those data models don't allow repeating or optional entities that aren't named and unitary.
>
>
> If we had allowed repeating anonymous groups, then if you interfaced DFDL directly to a language with tthis, we'd be having to generate names for these anonymous groups. We decided instead to keep the data model for DFDL simpler. If it repeats or is optional it has to be an element.
>
>
> The rationale here is the same reason we left out attributes of XSD. DFDL has only elements. This is because this dual-child tree where a node can have element children AND attribute children that have the same names that do not get mixed up... that's unique to XML, and DFDL is trying to be not so tied to XML/XSD, but able to describe data and project that data into the native data structures of many data models.
>
>
> ...mike beckerle
>
> Tresys Technology
>
>
> ________________________________
> From: Christofer Dutz <ch...@c-ware.de>
> Sent: Tuesday, January 15, 2019 4:15:26 PM
> To: dev@daffodil.apache.org
> Subject: Re: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
>
> (Wonder why every response turns out to be a private off-list response ... gotta remember to hit "reply to all")
>
> Thanks guys ... that worked like a charm ;-)
>
> Now I stumbled into the next little problem I couldn't find any documentation on.
>
> Now I have two situations:
> 1) The "parameterLength" provides the number of bytes all parameters consume, so the parser should continue parsing parameters as long as there are bytes left to read.
> How can I tell the sequence or the choice allow multiple instances?
>
> 2) I have "numItems" specifying the number of item-elements (also different types) so here not the number of bytes controls how many items are parsed, but the plain number of elements.
> Here too I cant find a way to specify a xs:minOccurs, xs:maxOccurs or dfdl:occursCount ... all seem to be invalid for both sequence and choice.
>
> Chris
>
> Am 15.01.19, 18:52 schrieb "Steve Lawrence" <sl...@apache.org>:
>
> It's not that Daffodil is ignoring dfdl:lengthKind="0", it's just that
> is allows length of zero to be valid. There are actually some use cases
> where zero length is valid, and you would want this to cause a backtrack
> if the children required more than zero bytes.
>
> In this case, you just need to make it so that if the length is zero
> then it does not attempt to parse any child elements. One way to
> accomplish this is via dfdl:occursCountKind="expression" and
> dfdl:occursCount, something like so:
>
> <xs:element name="payloads" minOccurs="0" maxOccurs="1
> dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes"
> dfdl:length="{../payloadsLength }"
> dfdl:occursCountKind="expression" dfdl:occursCount="{
> if (../payloadsLength eq 0) then 0 else 1
> }"
> <xs:complexType>
> <xs:sequence>
> <xs:choice>
> <xs:element ref="s7:S7RequestPayloadCpuServices"/>
> <xs:element ref="s7:S7RequestPayloadWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> This changes the payloads element to be optional (minOccurs=0) and
> defines the occurrences as either 0 or 1 based on the value of the
> payloadsLength. If the length is 1, the payloads element will not exist
> in the infoset and it will not attempt to parse the child data.
>
> - Steve
>
>
>
> On 1/15/19 10:39 AM, Christofer Dutz wrote:
> > Hi all,
> >
> > after working though the 6 tutorials on DFDL in general I think I have a much greater understanding on how I have to do things. I even managed to get my S7 protocol messages schema in a somewhat working condition.
> > Right now I’m having one problem:
> > A S7 Messages consists of a header, a number of variable length parameters and a number of variable length payloads.
> > As parameters and payloads are of variable length, the header contains a “parametersLength” and “payloadsLength” field which contains the number of bytes the parameters and payloads require in total.
> > So I defined something like this:
> >
> >
> > <xs:element name="parametersLength" type="s7:short"/>
> > <xs:element name="payloadsLength" type="s7:short"/>
> > <xs:element name="parameters" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}">
> > <xs:complexType>
> > <xs:sequence>
> > <xs:choice>
> > <xs:element ref="s7:S7GeneralParameterSetupCommunication"/>
> > <xs:element ref="s7:SS7RequestParameterCPUService"/>
> > <xs:element ref="s7:S7RequestParameterReadVar"/>
> > <xs:element ref="s7:S7RequestParameterWriteVar"/>
> > </xs:choice>
> > </xs:sequence>
> > </xs:complexType>
> > </xs:element>
> > <xs:element name="payloads" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}">
> > <xs:complexType>
> > <xs:sequence>
> > <xs:choice>
> > <xs:element ref="s7:S7RequestPayloadCpuServices"/>
> > <xs:element ref="s7:S7RequestPayloadWriteVar"/>
> > </xs:choice>
> > </xs:sequence>
> > </xs:complexType>
> > </xs:element>
> >
> > However in a request, that doesn’t contain any payloads, daffodil still tries to parse the payloads, even if the “length” of the sequence is set to explicit and to a length of 0 … why is it doing that?
> > Each parameter and payloads first byte contains the code that tells the parser what type it is and each of the elements in my schema use a discriminator to tell the parser which input it is requiring.
> >
> >
> > <xs:element name="S7RequestPayloadCpuServices">
> > <xs:annotation>
> > <xs:appinfo source="http://www.ogf.org/dfdl/">
> > <dfdl:discriminator test="{./type eq 0}"/>
> > </xs:appinfo>
> > </xs:annotation>
> > <xs:complexType>
> > <xs:sequence>
> > <xs:element name="type" type="s7:byte"/>
> > <xs:element name="transportSize" type="s7:byte"/><!-- fixed="9"-->
> > <xs:element name="length" type="s7:byte"/>
> > <xs:element name="sslId" type="s7:short"/>
> > <xs:element name="sslIndex" type="s7:short"/>
> > </xs:sequence>
> > </xs:complexType>
> > </xs:element>
> >
> > Hope it’s correct to do things like that …
> >
> > The effect is that Daffodil has correctly parsed all the parameters and as there is no payload (payloadLength = 0) it shouldn’t try to parse a payload, but it does and when reading the first byte it instantly fails as there is no data to parse anymore.
> >
> >
> > Chris
> >
>
>
>
>
>
>
>
Re: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi all,
I think I found a solution on my own ... the key is that I now output an payload element for every parameter, even if that is empty in some cases, with this, I was able to do the following:
<xs:element name="S7ResponseMessage">
<xs:complexType>
<xs:sequence>
<!-- Reserved value always 0x0000 -->
<xs:element name="reserved" type="s7:short" fixed="0"/>
<xs:element name="tpduReference" type="s7:short"/>
<xs:element name="parametersLength" type="s7:short"/>
<xs:element name="payloadsLength" type="s7:short"/>
<!-- UserData (type 7) responses don't have the error class and code -->
<xs:element name="errorClass" type="s7:byte" minOccurs="0"
dfdl:occursCountKind="expression" dfdl:occursCount="{if(../../type eq 3) then 1 else 0}"/>
<xs:element name="errorCode" type="s7:byte" minOccurs="0"
dfdl:occursCountKind="expression" dfdl:occursCount="{if(../../type eq 3) then 1 else 0}"/>
<xs:element name="parameters" minOccurs="0"
dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}"
dfdl:occursCountKind="expression" dfdl:occursCount="{if(../parametersLength gt 0) then 1 else 0}">
<xs:complexType>
<xs:sequence>
<xs:element name="parameter" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="type" type="s7:byte"/>
<xs:choice dfdl:choiceDispatchKey="{type}">
<xs:element dfdl:choiceBranchKey="240" ref="s7:S7GeneralParameterSetupCommunication"/>
<xs:element dfdl:choiceBranchKey="0" ref="s7:S7ResponseParameterCPUService"/>
<xs:element dfdl:choiceBranchKey="4" ref="s7:S7ResponseParameterReadVar"/>
<xs:element dfdl:choiceBranchKey="5" ref="s7:S7ResponseParameterWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="payloads" minOccurs="0"
dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}"
dfdl:occursCountKind="expression" dfdl:occursCount="{if((../payloadsLength gt 0) or (../parametersLength gt 0)) then 1 else 0}">
<xs:complexType>
<xs:sequence>
<xs:element name="payload" maxOccurs="unbounded"
dfdl:occursCountKind="expression" dfdl:occursCount="{fn:count(../../parameters[1]/parameter)}">
<xs:complexType>
<xs:sequence>
<xs:choice dfdl:choiceDispatchKey="{../../parameters[1]/parameter[fn:count(../payload)]/type}">
<xs:element dfdl:choiceBranchKey="240" ref="s7:S7GeneralPayloadSetupCommunication"/>
<xs:element dfdl:choiceBranchKey="0" ref="s7:S7ResponsePayloadCpuServices"/>
<xs:element dfdl:choiceBranchKey="4" ref="s7:S7ResponsePayloadReadVar"/>
<xs:element dfdl:choiceBranchKey="5" ref="s7:S7ResponsePayloadWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
So what I do now is that I use the type-element of the matching parameter for the second decision.
However I would like to report, das not even by debugging the code was I able to understand the error message about not supporting arrays.
"Query-style paths not supported. Must have '[...]' after array-element's name."
Cause the real reason is that the problem was that on the up-path only the last part can exist without the array notation and in this case the intermediate "parameters" fragment had to be added an array index.
How is the above solution performance-wise?
Chris
Am 16.01.19, 11:54 schrieb "Christofer Dutz" <ch...@c-ware.de>:
Hi all,
thanks for all of your help ... I am coming closer and closer to a first fully operational DFDL schema for S7 communication ... really looking forward to experimenting with that.
Now I ran into something probably quite special:
In my messages I have a header, parameters and payloads
While each payload has a "type" indicator, the payloads don't. The payloads are somewhat parsed by iterating over the previously parsed parameters and for each parameter (in exactly the same order) create a payload.
However not all parameters have payloads ...
So here's what I've come up so far:
<xs:element name="S7ResponseMessage">
<xs:complexType>
<xs:sequence>
<!-- Reserved value always 0x0000 -->
<xs:element name="reserved" type="s7:short" fixed="0"/>
<xs:element name="tpduReference" type="s7:short"/>
<xs:element name="parametersLength" type="s7:short"/>
<xs:element name="payloadsLength" type="s7:short"/>
<!-- UserData (type 7) responses don't have the error class and code -->
<xs:element name="errorClass" type="s7:byte" minOccurs="0"
dfdl:occursCountKind="expression" dfdl:occursCount="{if(../../type eq 3) then 1 else 0}"/>
<xs:element name="errorCode" type="s7:byte" minOccurs="0"
dfdl:occursCountKind="expression" dfdl:occursCount="{if(../../type eq 3) then 1 else 0}"/>
<xs:element name="parameters" minOccurs="0"
dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}"
dfdl:occursCountKind="expression" dfdl:occursCount="{if(../parametersLength gt 0) then 1 else 0}">
<xs:complexType>
<xs:sequence>
<xs:element name="parameter" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="type" type="s7:byte"/>
<xs:choice dfdl:choiceDispatchKey="{type}">
<xs:element dfdl:choiceBranchKey="240" ref="s7:S7GeneralParameterSetupCommunication"/>
<xs:element dfdl:choiceBranchKey="0" ref="s7:S7ResponseParameterCPUService"/>
<xs:element dfdl:choiceBranchKey="4" ref="s7:S7ResponseParameterReadVar"/>
<xs:element dfdl:choiceBranchKey="5" ref="s7:S7ResponseParameterWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="payloads" minOccurs="0"
dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}"
dfdl:occursCountKind="expression" dfdl:occursCount="{if(../payloadsLength gt 0) then 1 else 0}">
<xs:complexType>
<xs:sequence>
<xs:element name="payload" maxOccurs="unbounded"
dfdl:occursCountKind="expression" dfdl:occursCount="{count(../parameters/parameter)}">
<xs:complexType>
<!-- TODO: Somehow loop over the parameters and use those elements as type keys -->
<xs:sequence>
<xs:element name="type" type="s7:byte"/>
<xs:choice dfdl:choiceDispatchKey="{type}">
<xs:element dfdl:choiceBranchKey="0" ref="s7:S7ResponsePayloadCpuServices"/>
<xs:element dfdl:choiceBranchKey="4" ref="s7:S7ResponsePayloadReadVar"/>
<xs:element dfdl:choiceBranchKey="5" ref="s7:S7ResponsePayloadWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
So the "payload" element should occur exactly as often as there are parameters. Now I was thinking that if for a parameter there is no matching payload, I'll just not output any or just some dummy output.
The problem however is, how can I iterate over the parameters and for each get the "type" and depending on that select the right choice branch?
Chris
Am 16.01.19, 02:08 schrieb "Beckerle, Mike" <mb...@tresys.com>:
Christofer,
So in DFDL, unlike regular XSD, only elements can be repeating/optional.
You can't have max/minOccurs on a choice or sequence, only on an element.
DFDL isn't only for XML. Many data models that the DFDL infoset could be projected into, those data models don't allow repeating or optional entities that aren't named and unitary.
If we had allowed repeating anonymous groups, then if you interfaced DFDL directly to a language with tthis, we'd be having to generate names for these anonymous groups. We decided instead to keep the data model for DFDL simpler. If it repeats or is optional it has to be an element.
The rationale here is the same reason we left out attributes of XSD. DFDL has only elements. This is because this dual-child tree where a node can have element children AND attribute children that have the same names that do not get mixed up... that's unique to XML, and DFDL is trying to be not so tied to XML/XSD, but able to describe data and project that data into the native data structures of many data models.
...mike beckerle
Tresys Technology
________________________________
From: Christofer Dutz <ch...@c-ware.de>
Sent: Tuesday, January 15, 2019 4:15:26 PM
To: dev@daffodil.apache.org
Subject: Re: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
(Wonder why every response turns out to be a private off-list response ... gotta remember to hit "reply to all")
Thanks guys ... that worked like a charm ;-)
Now I stumbled into the next little problem I couldn't find any documentation on.
Now I have two situations:
1) The "parameterLength" provides the number of bytes all parameters consume, so the parser should continue parsing parameters as long as there are bytes left to read.
How can I tell the sequence or the choice allow multiple instances?
2) I have "numItems" specifying the number of item-elements (also different types) so here not the number of bytes controls how many items are parsed, but the plain number of elements.
Here too I cant find a way to specify a xs:minOccurs, xs:maxOccurs or dfdl:occursCount ... all seem to be invalid for both sequence and choice.
Chris
Am 15.01.19, 18:52 schrieb "Steve Lawrence" <sl...@apache.org>:
It's not that Daffodil is ignoring dfdl:lengthKind="0", it's just that
is allows length of zero to be valid. There are actually some use cases
where zero length is valid, and you would want this to cause a backtrack
if the children required more than zero bytes.
In this case, you just need to make it so that if the length is zero
then it does not attempt to parse any child elements. One way to
accomplish this is via dfdl:occursCountKind="expression" and
dfdl:occursCount, something like so:
<xs:element name="payloads" minOccurs="0" maxOccurs="1
dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes"
dfdl:length="{../payloadsLength }"
dfdl:occursCountKind="expression" dfdl:occursCount="{
if (../payloadsLength eq 0) then 0 else 1
}"
<xs:complexType>
<xs:sequence>
<xs:choice>
<xs:element ref="s7:S7RequestPayloadCpuServices"/>
<xs:element ref="s7:S7RequestPayloadWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
This changes the payloads element to be optional (minOccurs=0) and
defines the occurrences as either 0 or 1 based on the value of the
payloadsLength. If the length is 1, the payloads element will not exist
in the infoset and it will not attempt to parse the child data.
- Steve
On 1/15/19 10:39 AM, Christofer Dutz wrote:
> Hi all,
>
> after working though the 6 tutorials on DFDL in general I think I have a much greater understanding on how I have to do things. I even managed to get my S7 protocol messages schema in a somewhat working condition.
> Right now I’m having one problem:
> A S7 Messages consists of a header, a number of variable length parameters and a number of variable length payloads.
> As parameters and payloads are of variable length, the header contains a “parametersLength” and “payloadsLength” field which contains the number of bytes the parameters and payloads require in total.
> So I defined something like this:
>
>
> <xs:element name="parametersLength" type="s7:short"/>
> <xs:element name="payloadsLength" type="s7:short"/>
> <xs:element name="parameters" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}">
> <xs:complexType>
> <xs:sequence>
> <xs:choice>
> <xs:element ref="s7:S7GeneralParameterSetupCommunication"/>
> <xs:element ref="s7:SS7RequestParameterCPUService"/>
> <xs:element ref="s7:S7RequestParameterReadVar"/>
> <xs:element ref="s7:S7RequestParameterWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> <xs:element name="payloads" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}">
> <xs:complexType>
> <xs:sequence>
> <xs:choice>
> <xs:element ref="s7:S7RequestPayloadCpuServices"/>
> <xs:element ref="s7:S7RequestPayloadWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> However in a request, that doesn’t contain any payloads, daffodil still tries to parse the payloads, even if the “length” of the sequence is set to explicit and to a length of 0 … why is it doing that?
> Each parameter and payloads first byte contains the code that tells the parser what type it is and each of the elements in my schema use a discriminator to tell the parser which input it is requiring.
>
>
> <xs:element name="S7RequestPayloadCpuServices">
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/">
> <dfdl:discriminator test="{./type eq 0}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:complexType>
> <xs:sequence>
> <xs:element name="type" type="s7:byte"/>
> <xs:element name="transportSize" type="s7:byte"/><!-- fixed="9"-->
> <xs:element name="length" type="s7:byte"/>
> <xs:element name="sslId" type="s7:short"/>
> <xs:element name="sslIndex" type="s7:short"/>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> Hope it’s correct to do things like that …
>
> The effect is that Daffodil has correctly parsed all the parameters and as there is no payload (payloadLength = 0) it shouldn’t try to parse a payload, but it does and when reading the first byte it instantly fails as there is no data to parse anymore.
>
>
> Chris
>
Re: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi all,
thanks for all of your help ... I am coming closer and closer to a first fully operational DFDL schema for S7 communication ... really looking forward to experimenting with that.
Now I ran into something probably quite special:
In my messages I have a header, parameters and payloads
While each payload has a "type" indicator, the payloads don't. The payloads are somewhat parsed by iterating over the previously parsed parameters and for each parameter (in exactly the same order) create a payload.
However not all parameters have payloads ...
So here's what I've come up so far:
<xs:element name="S7ResponseMessage">
<xs:complexType>
<xs:sequence>
<!-- Reserved value always 0x0000 -->
<xs:element name="reserved" type="s7:short" fixed="0"/>
<xs:element name="tpduReference" type="s7:short"/>
<xs:element name="parametersLength" type="s7:short"/>
<xs:element name="payloadsLength" type="s7:short"/>
<!-- UserData (type 7) responses don't have the error class and code -->
<xs:element name="errorClass" type="s7:byte" minOccurs="0"
dfdl:occursCountKind="expression" dfdl:occursCount="{if(../../type eq 3) then 1 else 0}"/>
<xs:element name="errorCode" type="s7:byte" minOccurs="0"
dfdl:occursCountKind="expression" dfdl:occursCount="{if(../../type eq 3) then 1 else 0}"/>
<xs:element name="parameters" minOccurs="0"
dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}"
dfdl:occursCountKind="expression" dfdl:occursCount="{if(../parametersLength gt 0) then 1 else 0}">
<xs:complexType>
<xs:sequence>
<xs:element name="parameter" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="type" type="s7:byte"/>
<xs:choice dfdl:choiceDispatchKey="{type}">
<xs:element dfdl:choiceBranchKey="240" ref="s7:S7GeneralParameterSetupCommunication"/>
<xs:element dfdl:choiceBranchKey="0" ref="s7:S7ResponseParameterCPUService"/>
<xs:element dfdl:choiceBranchKey="4" ref="s7:S7ResponseParameterReadVar"/>
<xs:element dfdl:choiceBranchKey="5" ref="s7:S7ResponseParameterWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="payloads" minOccurs="0"
dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}"
dfdl:occursCountKind="expression" dfdl:occursCount="{if(../payloadsLength gt 0) then 1 else 0}">
<xs:complexType>
<xs:sequence>
<xs:element name="payload" maxOccurs="unbounded"
dfdl:occursCountKind="expression" dfdl:occursCount="{count(../parameters/parameter)}">
<xs:complexType>
<!-- TODO: Somehow loop over the parameters and use those elements as type keys -->
<xs:sequence>
<xs:element name="type" type="s7:byte"/>
<xs:choice dfdl:choiceDispatchKey="{type}">
<xs:element dfdl:choiceBranchKey="0" ref="s7:S7ResponsePayloadCpuServices"/>
<xs:element dfdl:choiceBranchKey="4" ref="s7:S7ResponsePayloadReadVar"/>
<xs:element dfdl:choiceBranchKey="5" ref="s7:S7ResponsePayloadWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
So the "payload" element should occur exactly as often as there are parameters. Now I was thinking that if for a parameter there is no matching payload, I'll just not output any or just some dummy output.
The problem however is, how can I iterate over the parameters and for each get the "type" and depending on that select the right choice branch?
Chris
Am 16.01.19, 02:08 schrieb "Beckerle, Mike" <mb...@tresys.com>:
Christofer,
So in DFDL, unlike regular XSD, only elements can be repeating/optional.
You can't have max/minOccurs on a choice or sequence, only on an element.
DFDL isn't only for XML. Many data models that the DFDL infoset could be projected into, those data models don't allow repeating or optional entities that aren't named and unitary.
If we had allowed repeating anonymous groups, then if you interfaced DFDL directly to a language with tthis, we'd be having to generate names for these anonymous groups. We decided instead to keep the data model for DFDL simpler. If it repeats or is optional it has to be an element.
The rationale here is the same reason we left out attributes of XSD. DFDL has only elements. This is because this dual-child tree where a node can have element children AND attribute children that have the same names that do not get mixed up... that's unique to XML, and DFDL is trying to be not so tied to XML/XSD, but able to describe data and project that data into the native data structures of many data models.
...mike beckerle
Tresys Technology
________________________________
From: Christofer Dutz <ch...@c-ware.de>
Sent: Tuesday, January 15, 2019 4:15:26 PM
To: dev@daffodil.apache.org
Subject: Re: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
(Wonder why every response turns out to be a private off-list response ... gotta remember to hit "reply to all")
Thanks guys ... that worked like a charm ;-)
Now I stumbled into the next little problem I couldn't find any documentation on.
Now I have two situations:
1) The "parameterLength" provides the number of bytes all parameters consume, so the parser should continue parsing parameters as long as there are bytes left to read.
How can I tell the sequence or the choice allow multiple instances?
2) I have "numItems" specifying the number of item-elements (also different types) so here not the number of bytes controls how many items are parsed, but the plain number of elements.
Here too I cant find a way to specify a xs:minOccurs, xs:maxOccurs or dfdl:occursCount ... all seem to be invalid for both sequence and choice.
Chris
Am 15.01.19, 18:52 schrieb "Steve Lawrence" <sl...@apache.org>:
It's not that Daffodil is ignoring dfdl:lengthKind="0", it's just that
is allows length of zero to be valid. There are actually some use cases
where zero length is valid, and you would want this to cause a backtrack
if the children required more than zero bytes.
In this case, you just need to make it so that if the length is zero
then it does not attempt to parse any child elements. One way to
accomplish this is via dfdl:occursCountKind="expression" and
dfdl:occursCount, something like so:
<xs:element name="payloads" minOccurs="0" maxOccurs="1
dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes"
dfdl:length="{../payloadsLength }"
dfdl:occursCountKind="expression" dfdl:occursCount="{
if (../payloadsLength eq 0) then 0 else 1
}"
<xs:complexType>
<xs:sequence>
<xs:choice>
<xs:element ref="s7:S7RequestPayloadCpuServices"/>
<xs:element ref="s7:S7RequestPayloadWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
This changes the payloads element to be optional (minOccurs=0) and
defines the occurrences as either 0 or 1 based on the value of the
payloadsLength. If the length is 1, the payloads element will not exist
in the infoset and it will not attempt to parse the child data.
- Steve
On 1/15/19 10:39 AM, Christofer Dutz wrote:
> Hi all,
>
> after working though the 6 tutorials on DFDL in general I think I have a much greater understanding on how I have to do things. I even managed to get my S7 protocol messages schema in a somewhat working condition.
> Right now I’m having one problem:
> A S7 Messages consists of a header, a number of variable length parameters and a number of variable length payloads.
> As parameters and payloads are of variable length, the header contains a “parametersLength” and “payloadsLength” field which contains the number of bytes the parameters and payloads require in total.
> So I defined something like this:
>
>
> <xs:element name="parametersLength" type="s7:short"/>
> <xs:element name="payloadsLength" type="s7:short"/>
> <xs:element name="parameters" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}">
> <xs:complexType>
> <xs:sequence>
> <xs:choice>
> <xs:element ref="s7:S7GeneralParameterSetupCommunication"/>
> <xs:element ref="s7:SS7RequestParameterCPUService"/>
> <xs:element ref="s7:S7RequestParameterReadVar"/>
> <xs:element ref="s7:S7RequestParameterWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> <xs:element name="payloads" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}">
> <xs:complexType>
> <xs:sequence>
> <xs:choice>
> <xs:element ref="s7:S7RequestPayloadCpuServices"/>
> <xs:element ref="s7:S7RequestPayloadWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> However in a request, that doesn’t contain any payloads, daffodil still tries to parse the payloads, even if the “length” of the sequence is set to explicit and to a length of 0 … why is it doing that?
> Each parameter and payloads first byte contains the code that tells the parser what type it is and each of the elements in my schema use a discriminator to tell the parser which input it is requiring.
>
>
> <xs:element name="S7RequestPayloadCpuServices">
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/">
> <dfdl:discriminator test="{./type eq 0}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:complexType>
> <xs:sequence>
> <xs:element name="type" type="s7:byte"/>
> <xs:element name="transportSize" type="s7:byte"/><!-- fixed="9"-->
> <xs:element name="length" type="s7:byte"/>
> <xs:element name="sslId" type="s7:short"/>
> <xs:element name="sslIndex" type="s7:short"/>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> Hope it’s correct to do things like that …
>
> The effect is that Daffodil has correctly parsed all the parameters and as there is no payload (payloadLength = 0) it shouldn’t try to parse a payload, but it does and when reading the first byte it instantly fails as there is no data to parse anymore.
>
>
> Chris
>
Re: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
Posted by "Beckerle, Mike" <mb...@tresys.com>.
Christofer,
So in DFDL, unlike regular XSD, only elements can be repeating/optional.
You can't have max/minOccurs on a choice or sequence, only on an element.
DFDL isn't only for XML. Many data models that the DFDL infoset could be projected into, those data models don't allow repeating or optional entities that aren't named and unitary.
If we had allowed repeating anonymous groups, then if you interfaced DFDL directly to a language with tthis, we'd be having to generate names for these anonymous groups. We decided instead to keep the data model for DFDL simpler. If it repeats or is optional it has to be an element.
The rationale here is the same reason we left out attributes of XSD. DFDL has only elements. This is because this dual-child tree where a node can have element children AND attribute children that have the same names that do not get mixed up... that's unique to XML, and DFDL is trying to be not so tied to XML/XSD, but able to describe data and project that data into the native data structures of many data models.
...mike beckerle
Tresys Technology
________________________________
From: Christofer Dutz <ch...@c-ware.de>
Sent: Tuesday, January 15, 2019 4:15:26 PM
To: dev@daffodil.apache.org
Subject: Re: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
(Wonder why every response turns out to be a private off-list response ... gotta remember to hit "reply to all")
Thanks guys ... that worked like a charm ;-)
Now I stumbled into the next little problem I couldn't find any documentation on.
Now I have two situations:
1) The "parameterLength" provides the number of bytes all parameters consume, so the parser should continue parsing parameters as long as there are bytes left to read.
How can I tell the sequence or the choice allow multiple instances?
2) I have "numItems" specifying the number of item-elements (also different types) so here not the number of bytes controls how many items are parsed, but the plain number of elements.
Here too I cant find a way to specify a xs:minOccurs, xs:maxOccurs or dfdl:occursCount ... all seem to be invalid for both sequence and choice.
Chris
Am 15.01.19, 18:52 schrieb "Steve Lawrence" <sl...@apache.org>:
It's not that Daffodil is ignoring dfdl:lengthKind="0", it's just that
is allows length of zero to be valid. There are actually some use cases
where zero length is valid, and you would want this to cause a backtrack
if the children required more than zero bytes.
In this case, you just need to make it so that if the length is zero
then it does not attempt to parse any child elements. One way to
accomplish this is via dfdl:occursCountKind="expression" and
dfdl:occursCount, something like so:
<xs:element name="payloads" minOccurs="0" maxOccurs="1
dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes"
dfdl:length="{../payloadsLength }"
dfdl:occursCountKind="expression" dfdl:occursCount="{
if (../payloadsLength eq 0) then 0 else 1
}"
<xs:complexType>
<xs:sequence>
<xs:choice>
<xs:element ref="s7:S7RequestPayloadCpuServices"/>
<xs:element ref="s7:S7RequestPayloadWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
This changes the payloads element to be optional (minOccurs=0) and
defines the occurrences as either 0 or 1 based on the value of the
payloadsLength. If the length is 1, the payloads element will not exist
in the infoset and it will not attempt to parse the child data.
- Steve
On 1/15/19 10:39 AM, Christofer Dutz wrote:
> Hi all,
>
> after working though the 6 tutorials on DFDL in general I think I have a much greater understanding on how I have to do things. I even managed to get my S7 protocol messages schema in a somewhat working condition.
> Right now I’m having one problem:
> A S7 Messages consists of a header, a number of variable length parameters and a number of variable length payloads.
> As parameters and payloads are of variable length, the header contains a “parametersLength” and “payloadsLength” field which contains the number of bytes the parameters and payloads require in total.
> So I defined something like this:
>
>
> <xs:element name="parametersLength" type="s7:short"/>
> <xs:element name="payloadsLength" type="s7:short"/>
> <xs:element name="parameters" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}">
> <xs:complexType>
> <xs:sequence>
> <xs:choice>
> <xs:element ref="s7:S7GeneralParameterSetupCommunication"/>
> <xs:element ref="s7:SS7RequestParameterCPUService"/>
> <xs:element ref="s7:S7RequestParameterReadVar"/>
> <xs:element ref="s7:S7RequestParameterWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> <xs:element name="payloads" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}">
> <xs:complexType>
> <xs:sequence>
> <xs:choice>
> <xs:element ref="s7:S7RequestPayloadCpuServices"/>
> <xs:element ref="s7:S7RequestPayloadWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> However in a request, that doesn’t contain any payloads, daffodil still tries to parse the payloads, even if the “length” of the sequence is set to explicit and to a length of 0 … why is it doing that?
> Each parameter and payloads first byte contains the code that tells the parser what type it is and each of the elements in my schema use a discriminator to tell the parser which input it is requiring.
>
>
> <xs:element name="S7RequestPayloadCpuServices">
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/">
> <dfdl:discriminator test="{./type eq 0}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:complexType>
> <xs:sequence>
> <xs:element name="type" type="s7:byte"/>
> <xs:element name="transportSize" type="s7:byte"/><!-- fixed="9"-->
> <xs:element name="length" type="s7:byte"/>
> <xs:element name="sslId" type="s7:short"/>
> <xs:element name="sslIndex" type="s7:short"/>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> Hope it’s correct to do things like that …
>
> The effect is that Daffodil has correctly parsed all the parameters and as there is no payload (payloadLength = 0) it shouldn’t try to parse a payload, but it does and when reading the first byte it instantly fails as there is no data to parse anymore.
>
>
> Chris
>
Re: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
Posted by Christofer Dutz <ch...@c-ware.de>.
(Wonder why every response turns out to be a private off-list response ... gotta remember to hit "reply to all")
Thanks guys ... that worked like a charm ;-)
Now I stumbled into the next little problem I couldn't find any documentation on.
Now I have two situations:
1) The "parameterLength" provides the number of bytes all parameters consume, so the parser should continue parsing parameters as long as there are bytes left to read.
How can I tell the sequence or the choice allow multiple instances?
2) I have "numItems" specifying the number of item-elements (also different types) so here not the number of bytes controls how many items are parsed, but the plain number of elements.
Here too I cant find a way to specify a xs:minOccurs, xs:maxOccurs or dfdl:occursCount ... all seem to be invalid for both sequence and choice.
Chris
Am 15.01.19, 18:52 schrieb "Steve Lawrence" <sl...@apache.org>:
It's not that Daffodil is ignoring dfdl:lengthKind="0", it's just that
is allows length of zero to be valid. There are actually some use cases
where zero length is valid, and you would want this to cause a backtrack
if the children required more than zero bytes.
In this case, you just need to make it so that if the length is zero
then it does not attempt to parse any child elements. One way to
accomplish this is via dfdl:occursCountKind="expression" and
dfdl:occursCount, something like so:
<xs:element name="payloads" minOccurs="0" maxOccurs="1
dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes"
dfdl:length="{../payloadsLength }"
dfdl:occursCountKind="expression" dfdl:occursCount="{
if (../payloadsLength eq 0) then 0 else 1
}"
<xs:complexType>
<xs:sequence>
<xs:choice>
<xs:element ref="s7:S7RequestPayloadCpuServices"/>
<xs:element ref="s7:S7RequestPayloadWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
This changes the payloads element to be optional (minOccurs=0) and
defines the occurrences as either 0 or 1 based on the value of the
payloadsLength. If the length is 1, the payloads element will not exist
in the infoset and it will not attempt to parse the child data.
- Steve
On 1/15/19 10:39 AM, Christofer Dutz wrote:
> Hi all,
>
> after working though the 6 tutorials on DFDL in general I think I have a much greater understanding on how I have to do things. I even managed to get my S7 protocol messages schema in a somewhat working condition.
> Right now I’m having one problem:
> A S7 Messages consists of a header, a number of variable length parameters and a number of variable length payloads.
> As parameters and payloads are of variable length, the header contains a “parametersLength” and “payloadsLength” field which contains the number of bytes the parameters and payloads require in total.
> So I defined something like this:
>
>
> <xs:element name="parametersLength" type="s7:short"/>
> <xs:element name="payloadsLength" type="s7:short"/>
> <xs:element name="parameters" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}">
> <xs:complexType>
> <xs:sequence>
> <xs:choice>
> <xs:element ref="s7:S7GeneralParameterSetupCommunication"/>
> <xs:element ref="s7:SS7RequestParameterCPUService"/>
> <xs:element ref="s7:S7RequestParameterReadVar"/>
> <xs:element ref="s7:S7RequestParameterWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> <xs:element name="payloads" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}">
> <xs:complexType>
> <xs:sequence>
> <xs:choice>
> <xs:element ref="s7:S7RequestPayloadCpuServices"/>
> <xs:element ref="s7:S7RequestPayloadWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> However in a request, that doesn’t contain any payloads, daffodil still tries to parse the payloads, even if the “length” of the sequence is set to explicit and to a length of 0 … why is it doing that?
> Each parameter and payloads first byte contains the code that tells the parser what type it is and each of the elements in my schema use a discriminator to tell the parser which input it is requiring.
>
>
> <xs:element name="S7RequestPayloadCpuServices">
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/">
> <dfdl:discriminator test="{./type eq 0}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:complexType>
> <xs:sequence>
> <xs:element name="type" type="s7:byte"/>
> <xs:element name="transportSize" type="s7:byte"/><!-- fixed="9"-->
> <xs:element name="length" type="s7:byte"/>
> <xs:element name="sslId" type="s7:short"/>
> <xs:element name="sslIndex" type="s7:short"/>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> Hope it’s correct to do things like that …
>
> The effect is that Daffodil has correctly parsed all the parameters and as there is no payload (payloadLength = 0) it shouldn’t try to parse a payload, but it does and when reading the first byte it instantly fails as there is no data to parse anymore.
>
>
> Chris
>
Re: Daffodil ignoring dfdl:length=0 when dfdl:lengthKind=explicit
Posted by Steve Lawrence <sl...@apache.org>.
It's not that Daffodil is ignoring dfdl:lengthKind="0", it's just that
is allows length of zero to be valid. There are actually some use cases
where zero length is valid, and you would want this to cause a backtrack
if the children required more than zero bytes.
In this case, you just need to make it so that if the length is zero
then it does not attempt to parse any child elements. One way to
accomplish this is via dfdl:occursCountKind="expression" and
dfdl:occursCount, something like so:
<xs:element name="payloads" minOccurs="0" maxOccurs="1
dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes"
dfdl:length="{../payloadsLength }"
dfdl:occursCountKind="expression" dfdl:occursCount="{
if (../payloadsLength eq 0) then 0 else 1
}"
<xs:complexType>
<xs:sequence>
<xs:choice>
<xs:element ref="s7:S7RequestPayloadCpuServices"/>
<xs:element ref="s7:S7RequestPayloadWriteVar"/>
</xs:choice>
</xs:sequence>
</xs:complexType>
</xs:element>
This changes the payloads element to be optional (minOccurs=0) and
defines the occurrences as either 0 or 1 based on the value of the
payloadsLength. If the length is 1, the payloads element will not exist
in the infoset and it will not attempt to parse the child data.
- Steve
On 1/15/19 10:39 AM, Christofer Dutz wrote:
> Hi all,
>
> after working though the 6 tutorials on DFDL in general I think I have a much greater understanding on how I have to do things. I even managed to get my S7 protocol messages schema in a somewhat working condition.
> Right now I’m having one problem:
> A S7 Messages consists of a header, a number of variable length parameters and a number of variable length payloads.
> As parameters and payloads are of variable length, the header contains a “parametersLength” and “payloadsLength” field which contains the number of bytes the parameters and payloads require in total.
> So I defined something like this:
>
>
> <xs:element name="parametersLength" type="s7:short"/>
> <xs:element name="payloadsLength" type="s7:short"/>
> <xs:element name="parameters" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../parametersLength}">
> <xs:complexType>
> <xs:sequence>
> <xs:choice>
> <xs:element ref="s7:S7GeneralParameterSetupCommunication"/>
> <xs:element ref="s7:SS7RequestParameterCPUService"/>
> <xs:element ref="s7:S7RequestParameterReadVar"/>
> <xs:element ref="s7:S7RequestParameterWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
> <xs:element name="payloads" dfdl:lengthKind="explicit" dfdl:lengthUnits="bytes" dfdl:length="{../payloadsLength}">
> <xs:complexType>
> <xs:sequence>
> <xs:choice>
> <xs:element ref="s7:S7RequestPayloadCpuServices"/>
> <xs:element ref="s7:S7RequestPayloadWriteVar"/>
> </xs:choice>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> However in a request, that doesn’t contain any payloads, daffodil still tries to parse the payloads, even if the “length” of the sequence is set to explicit and to a length of 0 … why is it doing that?
> Each parameter and payloads first byte contains the code that tells the parser what type it is and each of the elements in my schema use a discriminator to tell the parser which input it is requiring.
>
>
> <xs:element name="S7RequestPayloadCpuServices">
> <xs:annotation>
> <xs:appinfo source="http://www.ogf.org/dfdl/">
> <dfdl:discriminator test="{./type eq 0}"/>
> </xs:appinfo>
> </xs:annotation>
> <xs:complexType>
> <xs:sequence>
> <xs:element name="type" type="s7:byte"/>
> <xs:element name="transportSize" type="s7:byte"/><!-- fixed="9"-->
> <xs:element name="length" type="s7:byte"/>
> <xs:element name="sslId" type="s7:short"/>
> <xs:element name="sslIndex" type="s7:short"/>
> </xs:sequence>
> </xs:complexType>
> </xs:element>
>
> Hope it’s correct to do things like that …
>
> The effect is that Daffodil has correctly parsed all the parameters and as there is no payload (payloadLength = 0) it shouldn’t try to parse a payload, but it does and when reading the first byte it instantly fails as there is no data to parse anymore.
>
>
> Chris
>