You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by "Interrante, John A (GE Research, US)" <Jo...@ge.com> on 2022/01/25 18:09:28 UTC
RE: [jira] [Commented] (DAFFODIL-567) lengthKind="endOfParent" for simpleTypes
I was just looking at this bug yesterday since I'd found it after reading that Daffodil doesn't implement endOfParent yet (https://daffodil.apache.org/unsupported/), which'd disappointed me since I'd wanted to use endOfParent in a complicated DFDL schema I'm trying to simplify enough for runtime2 to support it (no dfdl:inputValueCalc or dfdl:outputValueCalc, only dfdl:length=expr at most).
The bug doesn't say how hard it would be to add support for endOfParent for both simpleTypes and complexTypes. I get it that endOfParent was added to the DFDL spec only after Daffodil had already been implemented and that endOfParent was too low priority until now which implies that the work is non-trivial (or it'd probably have been done as soon as endOfParent was added to the spec). How hard is it?
John
-----Original Message-----
From: Mike Beckerle (Jira) <ji...@apache.org>
Sent: Tuesday, January 25, 2022 7:56 AM
To: commits@daffodil.apache.org
Subject: EXT: [jira] [Commented] (DAFFODIL-567) lengthKind="endOfParent" for simpleTypes
WARNING: This email originated from outside of GE. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
[ https://issues.apache.org/jira/browse/DAFFODIL-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481912#comment-17481912 ]
Mike Beckerle commented on DAFFODIL-567:
----------------------------------------
Increased priority, as hacks like delimited hexBinary, with no delimiter specified, so as to simulate end-of-parent behavior, are proliferating.
One result of the lack of endOfParent, is such time as we do implement endOfParent, we probably want to make delimited with no delimiter specified (in scope) an SDE or at least SDW. (I will clarify with the DFDL workgroup if this is an SDE or not. The DFDL spec is not clear on this. )
However, this will have to be under control of an option/flag because many schemas may be depending on this delimited-with-no-delimiter behavior.
> lengthKind="endOfParent" for simpleTypes
> ----------------------------------------
>
> Key: DAFFODIL-567
> URL: https://issues.apache.org/jira/browse/DAFFODIL-567
> Project: Daffodil
> Issue Type: New Feature
> Components: Back End, DFDL Language
> Affects Versions: s7
> Reporter: Mike Beckerle
> Priority: Major
>
> See also DFDL-238 which is end of parent for complex types.
> See also DFDL-566 which is a bug related to end-of-parent
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
Re: [jira] [Commented] (DAFFODIL-567) lengthKind="endOfParent" for simpleTypes
Posted by Steve Lawrence <sl...@apache.org>.
I haven't investigated it much, but my initial thinking is that it's
actually not anything too different from what we already have, so
probably isn't too difficult.
We already have logic for specified length complex elements to set a
length limit for children (using withBitLengthLimit), so new endOfParent
parsers really just need to subtract the current bit position from that
limit to get the endOfParent length, rather than via an expression or
hard coded value. So calculating the length should be fairly
straightforward.
And many of our parsers are abstract classes, so a new endOfParent
parser would just need to extend the right class and implement a new
getBitLength function with the mentioned logic, and it should just work.
Though some parsers take a different approach and use Evaluatables, so
they would need a custom Evaluatable rather than a function override.
And some might even use different approaches as Daffodil has evolved
over time.
I think that is where things get a bit tedious, with just creating all
the parsers that need to use this new logic, and dealing with slightly
different implementation details of the base classes, and adding tests
to make sure all the cases are covered.
I also think there is some complexity in implementing many of the
limitations. There's about a dozen or so cases with this property that
lead to a schema definition error that must be checked for, some are
easier than others.
So algorithmically I don't think there is anything too hard or
different, with lots of reuse being likely. But just the number of
things to implement can get quite large. But this could be done
piecemeal, doing one parser at a time until they are all implemented
(e.g. first do binary numbers, then text numbers, then strings, etc.)
On 1/25/22 1:09 PM, Interrante, John A (GE Research, US) wrote:
> I was just looking at this bug yesterday since I'd found it after reading that Daffodil doesn't implement endOfParent yet (https://daffodil.apache.org/unsupported/), which'd disappointed me since I'd wanted to use endOfParent in a complicated DFDL schema I'm trying to simplify enough for runtime2 to support it (no dfdl:inputValueCalc or dfdl:outputValueCalc, only dfdl:length=expr at most).
>
> The bug doesn't say how hard it would be to add support for endOfParent for both simpleTypes and complexTypes. I get it that endOfParent was added to the DFDL spec only after Daffodil had already been implemented and that endOfParent was too low priority until now which implies that the work is non-trivial (or it'd probably have been done as soon as endOfParent was added to the spec). How hard is it?
>
> John
>
> -----Original Message-----
> From: Mike Beckerle (Jira) <ji...@apache.org>
> Sent: Tuesday, January 25, 2022 7:56 AM
> To: commits@daffodil.apache.org
> Subject: EXT: [jira] [Commented] (DAFFODIL-567) lengthKind="endOfParent" for simpleTypes
>
> WARNING: This email originated from outside of GE. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
>
>
> [ https://issues.apache.org/jira/browse/DAFFODIL-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481912#comment-17481912 ]
>
> Mike Beckerle commented on DAFFODIL-567:
> ----------------------------------------
>
> Increased priority, as hacks like delimited hexBinary, with no delimiter specified, so as to simulate end-of-parent behavior, are proliferating.
>
> One result of the lack of endOfParent, is such time as we do implement endOfParent, we probably want to make delimited with no delimiter specified (in scope) an SDE or at least SDW. (I will clarify with the DFDL workgroup if this is an SDE or not. The DFDL spec is not clear on this. )
>
> However, this will have to be under control of an option/flag because many schemas may be depending on this delimited-with-no-delimiter behavior.
>
>> lengthKind="endOfParent" for simpleTypes
>> ----------------------------------------
>>
>> Key: DAFFODIL-567
>> URL: https://issues.apache.org/jira/browse/DAFFODIL-567
>> Project: Daffodil
>> Issue Type: New Feature
>> Components: Back End, DFDL Language
>> Affects Versions: s7
>> Reporter: Mike Beckerle
>> Priority: Major
>>
>> See also DFDL-238 which is end of parent for complex types.
>> See also DFDL-566 which is a bug related to end-of-parent
>
>
>
> --
> This message was sent by Atlassian Jira
> (v8.20.1#820001)