You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by GitBox <gi...@apache.org> on 2020/05/01 20:27:21 UTC
[GitHub] [incubator-daffodil] jadams-tresys opened a new pull request #375: Fixed issues with resetting variables
jadams-tresys opened a new pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375
There were a few issues involving resetting variables that were
discovered in some schema projects. Issues have been fixed and test
cases have been added to the test suite to verify the fixes.
DAFFODIL-2334
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] stevedlawrence commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419384767
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
##########
@@ -289,7 +297,10 @@ final class PState private (
* just to track that the variable was read in case we need to backtrack.
*/
def variableRead(vrd: VariableRuntimeData) {
- changedVariablesStack.top += vrd.globalQName
+ val maybeVar = variableMap.find(vrd.globalQName)
+ Assert.invariant(maybeVar.isDefined)
+ if (maybeVar.get.state != VariableRead)
+ changedVariablesStack.top += vrd.globalQName
Review comment:
This means that there's now strict ordering between readVariable and variableRead (confusingly named, maybe a namechange is due?), correct? This function must always be called before readVariable, otherwise readVariable will change the state to VariableRead and then this will never add anything to the stack? This feels a bit fragile--I could easily see someone changing the order for some reason and then things break. Is there maybe a way to call variableRead from readVariable?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] stevedlawrence commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419378478
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/RuntimeData.scala
##########
@@ -994,6 +994,6 @@ final class VariableRuntimeData(
}
}
- def newVariableInstance: VariableInstance = VariableInstance(state, value, this, maybeDefaultValueExpr)
+ def newVariableInstance: VariableInstance = VariableInstance(state, value, this, maybeDefaultValueExpr, state, value)
Review comment:
This isn't clear to me. Why is state and value passed in as both the current state/value and the prior state/value? Seems to me when we create a new variable instance we want the prior state/value to be undefined/no value.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] mbeckerle commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
mbeckerle commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r418724033
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
##########
@@ -220,6 +221,7 @@ final class PState private (
if (variable.isDefined)
variable.get.reset
}}
+ changedVariablesStack.top.clear
Review comment:
So we're back to having to clear this again now. Please add comment as to why? Not obvious to me that we need this.
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/SequenceParserBases.scala
##########
@@ -332,6 +331,8 @@ abstract class SequenceParserBase(
case other => Assert.invariantFailed("Unexpected parse attempt status: " + other)
}
+ if (hasPoU) pstate.popPointOfUncertainty
Review comment:
This looks like the unwinding bug fix. Yes?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] jadams-tresys commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
jadams-tresys commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419451202
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
##########
@@ -289,7 +297,10 @@ final class PState private (
* just to track that the variable was read in case we need to backtrack.
*/
def variableRead(vrd: VariableRuntimeData) {
- changedVariablesStack.top += vrd.globalQName
+ val maybeVar = variableMap.find(vrd.globalQName)
+ Assert.invariant(maybeVar.isDefined)
+ if (maybeVar.get.state != VariableRead)
+ changedVariablesStack.top += vrd.globalQName
Review comment:
The problem with calling PState from VariableMap.readVariable() is that it doesn't know anything about PState. I thought we might be able to do a referringContext.isInstanceOf[PState], but it is always either a DPathCompileInfo or DPathElementCompileInfo. I think the call will need to remain strictly ordered and from VRef()
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] jadams-tresys commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
jadams-tresys commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419413379
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
##########
@@ -289,7 +297,10 @@ final class PState private (
* just to track that the variable was read in case we need to backtrack.
*/
def variableRead(vrd: VariableRuntimeData) {
- changedVariablesStack.top += vrd.globalQName
+ val maybeVar = variableMap.find(vrd.globalQName)
+ Assert.invariant(maybeVar.isDefined)
+ if (maybeVar.get.state != VariableRead)
+ changedVariablesStack.top += vrd.globalQName
Review comment:
Yes, there is strict ordering for this as currently implemented. Having readVariable be managed by DState while setVariable and newVariableInstance are handled by PState makes things a bit ugly here. Would it perhaps be better to implement a corresponding readVariable call in PState and UState and have DState.VRef() call that instead?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] jadams-tresys commented on pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
jadams-tresys commented on pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#issuecomment-623526183
I believe I have addressed all the comments in the review and have hopefully written a more useful commit message.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] stevedlawrence commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419453947
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
##########
@@ -289,7 +297,10 @@ final class PState private (
* just to track that the variable was read in case we need to backtrack.
*/
def variableRead(vrd: VariableRuntimeData) {
- changedVariablesStack.top += vrd.globalQName
+ val maybeVar = variableMap.find(vrd.globalQName)
+ Assert.invariant(maybeVar.isDefined)
+ if (maybeVar.get.state != VariableRead)
+ changedVariablesStack.top += vrd.globalQName
Review comment:
I don't feel strongly one way or the other, but you could modify readVariable to take in ParseOrUnparseState as a parameter, which VRef has access to from the DState. I'm not sure if that's any better or worse than other options, but that's the way to do it for that option.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] stevedlawrence commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419418343
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
##########
@@ -289,7 +297,10 @@ final class PState private (
* just to track that the variable was read in case we need to backtrack.
*/
def variableRead(vrd: VariableRuntimeData) {
- changedVariablesStack.top += vrd.globalQName
+ val maybeVar = variableMap.find(vrd.globalQName)
+ Assert.invariant(maybeVar.isDefined)
+ if (maybeVar.get.state != VariableRead)
+ changedVariablesStack.top += vrd.globalQName
Review comment:
Yeah, I think that would clean things up a bit. Thoughts on calling the PState/UState function from readVariable, since readVariable actually knows when variable state changes to VariableRead, since a read doesnt always modify that state. That would avoid this VariableRead check.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] jadams-tresys commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
jadams-tresys commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419403500
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/RuntimeData.scala
##########
@@ -994,6 +994,6 @@ final class VariableRuntimeData(
}
}
- def newVariableInstance: VariableInstance = VariableInstance(state, value, this, maybeDefaultValueExpr)
+ def newVariableInstance: VariableInstance = VariableInstance(state, value, this, maybeDefaultValueExpr, state, value)
Review comment:
So this "newVariableInstance" is confusingly what creates the initial VariableInstace. It isn't really related to the feature NVI outside of being called to create a VariableInstance object. It is called whenever a variable is created. I think renaming this to createVariableInstance may relieve the confusion. As for the prior state/value I think that may have been a change I left in while debugging some of the issues. I am removing it now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] jadams-tresys commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
jadams-tresys commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419030935
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/SequenceParserBases.scala
##########
@@ -332,6 +331,8 @@ abstract class SequenceParserBase(
case other => Assert.invariantFailed("Unexpected parse attempt status: " + other)
}
+ if (hasPoU) pstate.popPointOfUncertainty
Review comment:
Yes, we were popping the POU stack before calling PState.reset() and thus the PState.reset() was resetting variables outside of the POU.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] mbeckerle commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
mbeckerle commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419435302
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
##########
@@ -289,7 +297,10 @@ final class PState private (
* just to track that the variable was read in case we need to backtrack.
*/
def variableRead(vrd: VariableRuntimeData) {
- changedVariablesStack.top += vrd.globalQName
+ val maybeVar = variableMap.find(vrd.globalQName)
+ Assert.invariant(maybeVar.isDefined)
+ if (maybeVar.get.state != VariableRead)
+ changedVariablesStack.top += vrd.globalQName
Review comment:
When these sorts of ordering things come up, scaladoc helps, but I always try to put in some sort of a Assert.precondition() that is true only if things are being called in the right order.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] jadams-tresys commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
jadams-tresys commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419030815
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
##########
@@ -220,6 +221,7 @@ final class PState private (
if (variable.isDefined)
variable.get.reset
}}
+ changedVariablesStack.top.clear
Review comment:
I have now added a comment to this. The reason why it needs to be cleared is that when parsing a choice (or potentially an unordered sequence) there can be multiple calls to PState.reset() within the same push/popPointOfUncertainty. Which means that the list on changedVariablesStack.top will continue to accumulate changes if the list is not cleared. For example, choice branch 1 reads a variable but ultimately fails calling PState.reset(), which will reset the read on the variable, but then the choice enters branch 2 without popping the POU stack. Branch 2 then sets the variable but ultimately fails too and when PState.reset() gets called it will reset the setVariable call and then try to reset the readVariable call from branch 1 since it hasn't been cleared and we are still in the same POU.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] jadams-tresys commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
jadams-tresys commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419437137
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
##########
@@ -289,7 +297,10 @@ final class PState private (
* just to track that the variable was read in case we need to backtrack.
*/
def variableRead(vrd: VariableRuntimeData) {
- changedVariablesStack.top += vrd.globalQName
+ val maybeVar = variableMap.find(vrd.globalQName)
+ Assert.invariant(maybeVar.isDefined)
+ if (maybeVar.get.state != VariableRead)
+ changedVariablesStack.top += vrd.globalQName
Review comment:
Ok, will do.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-daffodil] jadams-tresys commented on a change in pull request #375: Fixed issues with resetting variables
Posted by GitBox <gi...@apache.org>.
jadams-tresys commented on a change in pull request #375:
URL: https://github.com/apache/incubator-daffodil/pull/375#discussion_r419458172
##########
File path: daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/parsers/PState.scala
##########
@@ -289,7 +297,10 @@ final class PState private (
* just to track that the variable was read in case we need to backtrack.
*/
def variableRead(vrd: VariableRuntimeData) {
- changedVariablesStack.top += vrd.globalQName
+ val maybeVar = variableMap.find(vrd.globalQName)
+ Assert.invariant(maybeVar.isDefined)
+ if (maybeVar.get.state != VariableRead)
+ changedVariablesStack.top += vrd.globalQName
Review comment:
Not sure why I didn' t think of that. This strange connection between DState and PState with the variableMap is confusing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org