You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by "John Interrante (Jira)" <ji...@apache.org> on 2022/05/17 15:21:00 UTC
[jira] [Comment Edited] (DAFFODIL-2697) Can the CLI pass a tunables object to the Runner?
[ https://issues.apache.org/jira/browse/DAFFODIL-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17538278#comment-17538278 ]
John Interrante edited comment on DAFFODIL-2697 at 5/17/22 3:20 PM:
--------------------------------------------------------------------
I agree that the goal of a TDML file should be to fully specify all the inputs/outputs/tunables of a test, so we shouldn't add CLI test options `-c, --config <file>` or `-Ttunable=value`. Instead, I would like to add a CLI test option `-I, --implementation <name>` where name is one of "daffodil", "daffodil-runtime2", or "ibm". The letter `-i` is already taken for the CLI test option `info`, but the only use of the letter `-I` is for the CLI parse option `infoset-type` so I'm fine with using `-I` for the CLI test option `-implementation` too (users should see a symmetry there).
Continuing the thought that we should not have to change a TDML test to change the implementation used to run it, I would also remove the tunable parameter `tdmlImplementation` and change my existing runtime2 test Scala files to pass the implementation argument "daffodil-runtime2" to Runner's apply (factory) function as its third String argument (the new String parameter would have the default value "daffodil" to avoid changing all other calls). In cases where I want to run the same test cases against both runtime1 and runtime2 to verify compatibility, I would define two Runners although I suppose we could also pass the implementation argument "daffodil-runtime2" to a runOneTest (per-testcase) function to override the Runner apply function's "daffodil" default for a single test case.
We may not have much need for an environment variable right now since we can't run existing daffodil TDML tests with runtime2 until we add runtime2 to Runner.defaultImplementationsDefaultDefault which is currently set to Seq("daffodil", "ibm") and extend runtime2 to handle DFDL expressions, etc. Still, I like the idea of an environment variable too. If we could get both runtime2 and ibm to work out of the box, setting DFDL_IMPLEMENTATION or TDML_IMPLEMENTATION (which sounds better to you?) to "daffodil-runtime2" or "ibm" and then running "sbt test" would be a LOT easier than following all the steps in ibmDFDLCrossTester/README.md. TDMLRunner.scala dynamically loads the implementation with the following match expression, so we might find a way to load the ibm implementation the same way as runtime2 too:
{code:java}
lazy val tdmlDFDLProcessorFactory: AbstractTDMLDFDLProcessorFactory = {
// tdmlImplementation is a tunable choice with three values.
val className = tunableObj.tdmlImplementation match {
// Right now daffodil and ibm use the same ProcessFactory name
case "daffodil" | "ibm" => "org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory"
case "daffodil-runtime2" => "org.apache.daffodil.tdml.processor.Runtime2TDMLDFDLProcessorFactory"
case other => Assert.invariantFailed("'%s' not valid for tdmlImplementation".format(other))
}
val clazz = Try(Class.forName(className))
// rest elided...{code}
was (Author: jinterrante):
I agree that the goal of a TDML file should be to fully specify all the inputs/outputs/tunables of a test, so we shouldn't add CLI test options `-c, --config <file>` or `-Ttunable=value`. Instead, I would like to add a CLI test option `-I, --implementation <name>` where name is one of "daffodil", "daffodil-runtime2", or "ibm". The letter `-i` is already taken for the CLI test option `--info`, but the only use of the letter `-I` is for the CLI parse option `--infoset-type` so I'm fine with using `-I` for the CLI test option `--implementation` too (users should see a symmetry there).
Continuing the thought that we should not have to change a TDML test to change the implementation used to run it, I would also remove the tunable parameter `tdmlImplementation` and change my existing runtime2 test Scala files to pass the implementation argument "daffodil-runtime2" to Runner's apply (factory) function as its third String argument (the new String parameter would have the default value "daffodil" to avoid changing all other calls). In cases where I want to run the same test cases against both runtime1 and runtime2 to verify compatibility, I would define two Runners although I suppose we could also pass the implementation argument "daffodil-runtime2" to a runOneTest (per-testcase) function to override the Runner apply function's "daffodil" default for a single test case.
We may not have much need for an environment variable right now since we can't run existing daffodil TDML tests with runtime2 until we add runtime2 to Runner.defaultImplementationsDefaultDefault which is currently set to Seq("daffodil", "ibm") and extend runtime2 to handle DFDL expressions, etc. Still, I like the idea of an environment variable too. If we could get both runtime2 and ibm to work out of the box, setting DFDL_IMPLEMENTATION or TDML_IMPLEMENTATION (which sounds better to you?) to "daffodil-runtime2" or "ibm" and then running "sbt test" would be a LOT easier than following all the steps in ibmDFDLCrossTester/README.md. TDMLRunner.scala dynamically loads the implementation with the following match expression, so we might find a way to load the ibm implementation the same way as runtime2 too:
{code:java}
lazy val tdmlDFDLProcessorFactory: AbstractTDMLDFDLProcessorFactory = {
// tdmlImplementation is a tunable choice with three values.
val className = tunableObj.tdmlImplementation match {
// Right now daffodil and ibm use the same ProcessFactory name
case "daffodil" | "ibm" => "org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory"
case "daffodil-runtime2" => "org.apache.daffodil.tdml.processor.Runtime2TDMLDFDLProcessorFactory"
case other => Assert.invariantFailed("'%s' not valid for tdmlImplementation".format(other))
}
val clazz = Try(Class.forName(className))
// rest elided...{code}
> Can the CLI pass a tunables object to the Runner?
> -------------------------------------------------
>
> Key: DAFFODIL-2697
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2697
> Project: Daffodil
> Issue Type: Improvement
> Components: CLI, TDML Runner
> Reporter: John Interrante
> Priority: Minor
>
> It would be very nice if we could run the same TDML file with either runtime1 or runtime2 without changing the TDML file. Right now the only way to run a TDML file or test with runtime2 is to define a "runtime2" config (which could be a file or an embedded config) and use it in either defaultConfig="runtime2" on the TDML file or config="runtime2" on the TDML test. Is there any reason why the daffodil test subcommand can't accept the same `-c, --config <file>` and `-Ttunable=value` options as most of the other subcommands, build a tunables object, and pass it to the Runner? Most of the other subcommands pass a tunables object from the CLI to the rest of Daffodil, but I read a comment on DFDLTestSuite that says
> {noformat}
> * Keep this independent of Daffodil, so that it can be used to run tests against other DFDL implementations as well.
> * E.g., it should only need an API specified as a collection of Scala traits, and some simple way to inject
> * dependency on one factory to create processors.{noformat}
> Can the CLI still pass a tunables object explicitly to DFDLTestSuite anyway, or pass it implicitly somehow?
--
This message was sent by Atlassian Jira
(v8.20.7#820007)