You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@daffodil.apache.org by Christofer Dutz <ch...@c-ware.de> on 2019/02/06 19:14:45 UTC
Re: [TDML] Options to ignore values and ordering?
Hi all,
thanks for the tips.
I ended up creating a new Inputter:
package org.apache.plc4x.sandbox.java.s7.utils
import org.apache.commons.scxml2.Context
import org.apache.daffodil.dpath.NodeInfo
import org.apache.daffodil.japi.infoset.{InfosetInputterProxy, W3CDOMInfosetInputter}
class W3CDOMTemplateInfosetInputter(document: org.w3c.dom.Document, context: Context) extends InfosetInputterProxy {
override val infosetInputter = new W3CDOMInfosetInputter(document)
override def getSimpleText(primType: NodeInfo.Kind): String = {
var value = super.getSimpleText(primType)
if(value.startsWith("${") && value.endsWith("}")) {
val varName = value.substring(2, value.length - 1)
val varValue = context.get(varName)
if(varValue.isInstanceOf[String]) {
value = varValue.toString
}
}
value
}
}
With that in my XML I could just add “${variable-name}” like content and output that instead …
I solved the other way around (Saving things in the context) with simple Xpath … as I’m not really interested in the XML document itself.
So far my generic Daffodil and SCXML driver seems to be going nicely … really looking forward to finishing it and doing some benchmarking.
Chris
Von: "Beckerle, Mike" <mb...@tresys.com>
Antworten an: "users@daffodil.apache.org" <us...@daffodil.apache.org>
Datum: Mittwoch, 30. Januar 2019 um 01:29
An: "users@daffodil.apache.org" <us...@daffodil.apache.org>
Betreff: Re: [TDML] Options to ignore values and ordering?
It is possible to assemble a DFDL infoset programmatically.
The InfosetInputters do this in order to feed data from XML into the unparser.
This capability is not exposed for use from an external API for Daffodil like the JAPI/SAPI modules. The classes are there and can be used.
Take a look at the InfosetInputter class/trait and the XML and JSON flavors of it and see if it makes sense to you for constructing messages programmatically.
________________________________
From: Christofer Dutz <ch...@c-ware.de>
Sent: Tuesday, January 29, 2019 2:43:01 PM
To: users@daffodil.apache.org
Subject: Re: [TDML] Options to ignore values and ordering?
Hi Mike,
thanks for that input. However I guess that I don’t even need full verification. A set of XPath expressions should be enough.
The other direction however I think I could need some help with. Daffodil is able to serialize XML representations of my format.
However is there a way to not have an XML literal in the end but some mechanism that allows me to provide variable values?
I was thinking of something like this?
<state id="sendCotpConnectionRequest">
<onentry>
<plc4x:send>
<s7:tpktMessage>
<magicByte>3</magicByte>
<reserved>0</reserved>
<length>18</length>
<userData>
<headerLength>17</headerLength>
<type>224</type>
<CotpTpduConnectionRequest>
<destinationReference>1</destinationReference>
<!-- Insert the value for "cotp-local-reference" as short here -->
<sourceReference><plc4x:insert type="s7:short" name="cotp-local-reference"/></sourceReference>
<protocolClass>0</protocolClass>
<parameters>
<parameter>
<type>193</type>
<parameterLength>2</parameterLength>
<CotpParameterCallingTsap>
<tsapId><plc4x:insert type="s7:short" name="cotp-calling-tsap"/></tsapId>
</CotpParameterCallingTsap>
</parameter>
<parameter>
<type>194</type>
<parameterLength>2</parameterLength>
<CotpParameterCalledTsap>
<tsapId><plc4x:insert type="s7:short" name="cotp-called-tsap"/></tsapId>
</CotpParameterCalledTsap>
</parameter>
<parameter>
<type>192</type>
<parameterLength>1</parameterLength>
<CotpParameterTpduSize>
<tpduSize><plc4x:insert type="s7:byte" name="cotp-tpdu-size"/></tpduSize>
</CotpParameterTpduSize>
</parameter>
</parameters>
</CotpTpduConnectionRequest>
</userData>
</s7:tpktMessage>
</plc4x:send>
</onentry>
<transition target="receiveCotpConnectionResponse"/>
</state>
Would there be some way to extend Daffodil to do something likte that?
Chris
Von: "Beckerle, Mike" <mb...@tresys.com>
Antworten an: "users@daffodil.apache.org" <us...@daffodil.apache.org>
Datum: Dienstag, 29. Januar 2019 um 15:49
An: "users@daffodil.apache.org" <us...@daffodil.apache.org>
Betreff: Re: [TDML] Options to ignore values and ordering?
I think you are going to have to write a special XML comparison tool.
The unordered issue is usually handled by cannonicalization - i.e., sort things first, then verify the result of the sort against your expected data.
You can use the XML comparison tool we wrote for the Daffodil TDML runner as a starting point for yours if you want.
Look in XMLUtils.scala in daffodil-lib.
The usual trick with such comparison tools is getting out useful diagnostic info for when items are NOT equal, and not stopping at the first little inequality, but somehow "resynchronizing" back to the subsequent parts of the data that are still the same, so you can continue comparing more of the structure. This is not always possible (nor required for your use case perhaps) but is one of the bigger challenges.
I really recommend this approach of building a test rig like your own specialized TDML that exactly lets you express the tests you want, does the kind of comparison you want, etc. This was one of the really good decisions we made early on in Daffodil development and has paid off very well.
These kinds of comparison tools are very easy to write in Scala using its XML support. Some people in the scala community are suggesting switching to different XML libraries, but I have not found a need for that, and Daffodil makes very heavy use of the original scala.xml library.
________________________________
From: Christofer Dutz <ch...@c-ware.de>
Sent: Tuesday, January 29, 2019 7:29:43 AM
To: users@daffodil.apache.org
Subject: [TDML] Options to ignore values and ordering?
Hi all,
I am currently trying to use DFDL, to define the interaction in industrial protocols with DFDL and SCXML (Other thread).
Now I am defining in my SCXML document message content being sent via DFDL. However when defining the receiving part I would need something I could imagine is part of TDML.
In my case a response can contain a number of parameters. Each parameter having different payload. However the order of these is not pre-defined. I would like to do something like this:
<state id="receiveCotpConnectionResponse">
<onentry>
<plc4x:receive timeout="5000">
<s7:tpktMessage>
<magicByte>3</magicByte>
<reserved>0</reserved>
<!-- Just ignore the content of this field, we don't care about it as it's only required for parsing. -->
<length><plc4x:ignore/></length>
<userData>
<!-- Just ignore the content of this field, we don't care about it as it's only required for parsing. -->
<headerLength><plc4x:ignore/></headerLength>
<type>208</type>
<CotpTpduConnectionResponse>
<!-- Make sure the reply uses the same reference as we used in the request. -->
<destinationReference><plc4x:verify type="s7:short" name="cotp-local-reference"/></destinationReference>
<!-- Extract the reference the remote would like us to use in this session. -->
<sourceReference><plc4x:extract type="s7:short" name="cotp-remote-reference"/></sourceReference>
<protocolClass>0</protocolClass>
<parameters>
<!--
These elements might be transferred in alternate order, we just care about all of them being
transferred.
-->
<plc4x:unordered>
<parameter>
<type>192</type>
<parameterLength>1</parameterLength>
<CotpParameterTpduSize>
<tpduSize><plc4x:extract type="s7:byte" name="cotp-tpdu-size"/></tpduSize>
</CotpParameterTpduSize>
</parameter>
<parameter>
<type>193</type>
<parameterLength>2</parameterLength>
<CotpParameterCallingTsap>
<tsapId><plc4x:extract type="s7:short" name="cotp-calling-tsap"/></tsapId>
</CotpParameterCallingTsap>
</parameter>
<parameter>
<type>194</type>
<parameterLength>2</parameterLength>
<CotpParameterCalledTsap>
<tsapId><plc4x:extract type="s7:short" name="cotp-called-tsap"/></tsapId>
</CotpParameterCalledTsap>
</parameter>
<!-- The remote might be passing other parameters, we'll just ignore them for now -->
<plc4x:ignore/>
</plc4x:unordered>
</parameters>
</CotpTpduConnectionResponse>
</userData>
</s7:tpktMessage>
</plc4x:receive>
</onentry>
<transition cond="" target="sendS7SetupCommunicationRequest"/>
<transition cond="" target="error"/>
</state>
“s7” is the namespace of my DFDL schema. The default namespace is the namespace of SCXML. The “plc4x” namespace is sort of my idea what’s missing.
“insert” would insert some value from an internal context into the template, “extract” would take values from the message and save it in the context, “verify” would compare the value it reads with one in the context, “ignore” would just ignore it and it’s content, “unordered” would sort of allow me to expect a sequence of elements, but not necessarily in that order.
If TDML has any form of way to define such a “plc4x:ignore” or “plc4x:unordered”, I would really like to use that. At least those are the elements I could expect to be useful in something like TDML.
Any ideas/suggestions?
Chris