You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@daffodil.apache.org by "Costello, Roger L." <co...@mitre.org> on 2019/09/18 14:10:03 UTC
Everything you ever wanted to know about dfdl:initiator (but was
afraid to ask)
Hello DFDL community,
I think that I have now mastered dfdl:initiator. I think that I have turned every stone and now know everything there is to know about dfdl:initiator. Below are slides I created, which captures everything I know about dfdl:initiator. I am seeking your review of the slides. Are they clear and easy to understand? Do they have any mistakes, including typo mistakes? Are they complete - do they cover everything there is to know about dfdl:initiator? /Roger
[cid:image065.png@01D56E09.375C6220]
[cid:image066.png@01D56E09.375C6220]
[cid:image067.png@01D56E09.375C6220]
[cid:image068.png@01D56E09.375C6220]
[cid:image069.png@01D56E09.375C6220]
[cid:image070.png@01D56E09.375C6220]
[cid:image071.png@01D56E09.375C6220]
[cid:image072.png@01D56E09.375C6220]
[cid:image073.png@01D56E09.375C6220]
[cid:image074.png@01D56E09.375C6220]
[cid:image075.png@01D56E09.375C6220]
[cid:image076.png@01D56E09.375C6220]
[cid:image077.png@01D56E09.375C6220]
[cid:image078.png@01D56E09.375C6220]
[cid:image079.png@01D56E09.375C6220]
[cid:image080.png@01D56E09.375C6220]
[cid:image081.png@01D56E09.375C6220]
[cid:image082.png@01D56E09.375C6220]
[cid:image083.png@01D56E09.375C6220]
[cid:image084.png@01D56E09.375C6220]
[cid:image085.png@01D56E09.375C6220]
[cid:image086.png@01D56E09.375C6220]
[cid:image087.png@01D56E09.375C6220]
[cid:image088.png@01D56E09.375C6220]
[cid:image089.png@01D56E09.375C6220]
[cid:image090.png@01D56E09.375C6220]
[cid:image091.png@01D56E09.375C6220]
[cid:image092.png@01D56E09.375C6220]
[cid:image093.png@01D56E09.375C6220]
[cid:image094.png@01D56E09.375C6220]
[cid:image095.png@01D56E09.375C6220]
[cid:image096.png@01D56E09.375C6220]
Re: Everything you ever wanted to know about dfdl:initiator (but was
afraid to ask)
Posted by "Beckerle, Mike" <mb...@tresys.com>.
I put my feedback between the slides below. I've only done the initiator slides so far. I stopped where you introduce emptyValueDelimiterPolicy (EVDP).
This material is great.
________________________________
From: Costello, Roger L. <co...@mitre.org>
Sent: Wednesday, September 18, 2019 10:10 AM
To: users@daffodil.apache.org <us...@daffodil.apache.org>
Subject: Everything you ever wanted to know about dfdl:initiator (but was afraid to ask)
Hello DFDL community,
I think that I have now mastered dfdl:initiator. I think that I have turned every stone and now know everything there is to know about dfdl:initiator. Below are slides I created, which captures everything I know about dfdl:initiator. I am seeking your review of the slides. Are they clear and easy to understand? Do they have any mistakes, including typo mistakes? Are they complete – do they cover everything there is to know about dfdl:initiator? /Roger
[cid:image065.png@01D56E09.375C6220]
[cid:image066.png@01D56E09.375C6220]
Above, we call them "initiator", but people commonly refer to this concept as "tags", "labels", etc.
[cid:image067.png@01D56E09.375C6220]
[cid:image068.png@01D56E09.375C6220]
Above on your dfdl:initiatedContent slide, I think all children of the containing sequence/choice must have initiators. Your text says all "elements".
This is in general an issue: what to call the "things" that can be in a sequence/choice. Daffodil borrows XML Schema terminology and calls them "terms", but the DFDL spec doesn't consistently use this so has "element or model group" in many places.
You use the phrase "specify an empty initiator", to mean dfdl:initiator="".
I found this confusing.
First, what does it mean to "specify an initiator". It means the term has lexically in scope surrounding it, or lexically written on it, a binding for the dfdl:initiator property which is not "". If it has (in scope or on it) a dfdl:initiator="", that is "not having an initiator".
[cid:image069.png@01D56E09.375C6220]
More typically these elements with initiators would be minOccurs="0" (with dfdl:occursCountKind="implicit" in scope) so that the presence/absence of the initiator is used to determine whether they are present or not. That would really motivate your example better.
Probably 2 slides above where you explain initiatedContent, you should point out its usage is commonly for data that is optional so as to use the initiators to establish whether it is present or not.
[cid:image070.png@01D56E09.375C6220]
typically initiator="" would be inherited from the surrounding lexical context. The error message would be the same.
[cid:image071.png@01D56E09.375C6220]
[cid:image072.png@01D56E09.375C6220]
[cid:image073.png@01D56E09.375C6220]
[cid:image074.png@01D56E09.375C6220]
Note that you can't have %ES; in the choices for an initiator if dfdl:initiatedContent is "yes" because well %ES; means 'nothing there' and you can't detect something by looking for nothing.
[cid:image075.png@01D56E09.375C6220]
[cid:image076.png@01D56E09.375C6220]
The real examples I've seen of this sort of thing use computed delimiters to avoid the need for quoting/escaping.
E.g., suppose data will look like
[some stuff]
[other stuff]
[more stuff].
But now the stuff needs to contain "[" or "]". You can either get into quoting/escaping with escape schemes, or the other technique is commonly that each term will instead consist of two characters first, the initiator and terminator to be used, then the term itself, using that initiator/terminator.
So if the 2nd one contains "[", then the data could be
[][some stuff]
()(stuff with [ and ] in it)
[][more stuff]
[cid:image077.png@01D56E09.375C6220]
[cid:image078.png@01D56E09.375C6220]
Examples where the initiators are variants of line-endings.... This is super rare in my experience. Combinations like %WSP*;SomeText%WSP*; are common, but there is almost always some text content that is recognizable in an initiator.
A less contrived initiator that is all-whitespace (assuming you want such an example) would be a TAB character maybe.
[cid:image079.png@01D56E09.375C6220]
Your slide example below for "Initiator can be a byte value" doesn't use a byte value. It uses a numeric character entity. The difference here is important. %#83; is "S". In utf-16 that occupies 2 bytes.
If you want to look for an initiator that is a single byte say, 0xFF in the data, that's what byte-values are for, so even in a utf-16 text where the characters are 2-bytes each, dfdl:initiator="%#rFF;" means look for a single byte containing 0xFF.
[cid:image080.png@01D56E09.375C6220]
[cid:image081.png@01D56E09.375C6220]
[cid:image082.png@01D56E09.375C6220]
Same comment as above. Initiator of %NL; alone, ... never seen it.
[cid:image083.png@01D56E09.375C6220]
I would move the above slide earlier to dispense with this whole notion as early as possible.
[cid:image084.png@01D56E09.375C6220]
[cid:image085.png@01D56E09.375C6220]
On unparsing the case specified in the first initiator (remember initiator is a whitespace separated list) is used.
[cid:image086.png@01D56E09.375C6220]
[cid:image087.png@01D56E09.375C6220]
I will work on your slides about emptyValueDelimiterPolicy in a separate message. This is a super thorny topic. I'm not sure it can be dealt with separately from separatorSuppressionPolicy.
[cid:image088.png@01D56E09.375C6220]
[cid:image089.png@01D56E09.375C6220]
[cid:image090.png@01D56E09.375C6220]
[cid:image091.png@01D56E09.375C6220]
[cid:image092.png@01D56E09.375C6220]
[cid:image093.png@01D56E09.375C6220]
[cid:image094.png@01D56E09.375C6220]
[cid:image095.png@01D56E09.375C6220]
[cid:image096.png@01D56E09.375C6220]