You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by "Adams, Joshua" <ja...@owlcyberdefense.com> on 2020/10/29 20:07:39 UTC
Problems with expressions and variables
After spending a while banging my head against a wall as to why my simple test demonstrating newVariableInstance with a default value expression wasn't working, I think I understand what is going wrong now.
Here is a copy of the simple test case for newVariableInstance with a default value expression:
<xs:element name="nvi12">
<xs:complexType>
<xs:sequence>
<xs:element name="newDefaultValue" type="xsd:int" dfdl:lengthKind="explicit" dfdl:length="1" />
<xs:element name="innerSeq">
<xs:complexType>
<xs:sequence>
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:newVariableInstance ref="ex:myVar1"
defaultValue="{ ../ex:newDefaultValue }" />
</xs:appinfo>
</xs:annotation>
<xs:element name="varValue" type="xs:int"
dfdl:inputValueCalc="{ $ex:myVar1 }" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
I had added code to the VariableMap1 readVariable function to evaluate the expression if the variable didn't already have a value, so it was getting evaluated when element 'varValue' was being parsed. This expression evaluation then triggered the evaluation of the myVar1, which itself is an expression that then needs to be evaluated. The problem is when Daffodil attempts to run the expression for the newVariableInstance defaultValue, it does so from the context of 'varValue' and ultimately fails.
To me this seems like a problem that could potentially affect any expression that then references another non-constant expression. I took a quick look through our tests and I was unable to find a test case that covered this scenario. Is this something that Daffodil should support? I took a look through the spec and didn't find anything prohibiting this, and it seems like my test case would describe the typical use case for newVariableInstance referencing a non-constant expression.
Assuming this test is a valid use case for a default value as an expression, the issue becomes when should the expression be evaluated? This also could have issues that require some changes to the variable API. My initial approach I described earlier attempted to evaluate the expression when the variable was read, but that resulted in an incorrect context for evaluating the variable's expression. Another option would be to evaluate the expression when the VariableRuntimeData structure is created for the myVar1's newVariableInstance. This is the location where values are set for constant expressions, but the VariableRuntimeData class doesn't seem to have access to the PState or UState needed to evaluate non-constant expressions.
Just want to make sure that this is a valid test case and if so, what approach should be taken to get this working.
Josh
Re: Problems with expressions and variables
Posted by "Beckerle, Mike" <mb...@owlcyberdefense.com>.
The default value expression should be evaluated not on demand, but when the sequence is entered. That will give it the correct context.
The evaluation of the default value expression should work the same way as if it was in a setVariable annotation, with the exception that it sets the default value, not the assigned value.
________________________________
From: Adams, Joshua <ja...@owlcyberdefense.com>
Sent: Thursday, October 29, 2020 4:07:39 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: Problems with expressions and variables
After spending a while banging my head against a wall as to why my simple test demonstrating newVariableInstance with a default value expression wasn't working, I think I understand what is going wrong now.
Here is a copy of the simple test case for newVariableInstance with a default value expression:
<xs:element name="nvi12">
<xs:complexType>
<xs:sequence>
<xs:element name="newDefaultValue" type="xsd:int" dfdl:lengthKind="explicit" dfdl:length="1" />
<xs:element name="innerSeq">
<xs:complexType>
<xs:sequence>
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:newVariableInstance ref="ex:myVar1"
defaultValue="{ ../ex:newDefaultValue }" />
</xs:appinfo>
</xs:annotation>
<xs:element name="varValue" type="xs:int"
dfdl:inputValueCalc="{ $ex:myVar1 }" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
I had added code to the VariableMap1 readVariable function to evaluate the expression if the variable didn't already have a value, so it was getting evaluated when element 'varValue' was being parsed. This expression evaluation then triggered the evaluation of the myVar1, which itself is an expression that then needs to be evaluated. The problem is when Daffodil attempts to run the expression for the newVariableInstance defaultValue, it does so from the context of 'varValue' and ultimately fails.
To me this seems like a problem that could potentially affect any expression that then references another non-constant expression. I took a quick look through our tests and I was unable to find a test case that covered this scenario. Is this something that Daffodil should support? I took a look through the spec and didn't find anything prohibiting this, and it seems like my test case would describe the typical use case for newVariableInstance referencing a non-constant expression.
Assuming this test is a valid use case for a default value as an expression, the issue becomes when should the expression be evaluated? This also could have issues that require some changes to the variable API. My initial approach I described earlier attempted to evaluate the expression when the variable was read, but that resulted in an incorrect context for evaluating the variable's expression. Another option would be to evaluate the expression when the VariableRuntimeData structure is created for the myVar1's newVariableInstance. This is the location where values are set for constant expressions, but the VariableRuntimeData class doesn't seem to have access to the PState or UState needed to evaluate non-constant expressions.
Just want to make sure that this is a valid test case and if so, what approach should be taken to get this working.
Josh