[GitHub] [daffodil] stevedlawrence commented on a diff in pull request #831: Added TDML with precompiled binary schema feature.

[GitBox <gi...@apache.org>]

stevedlawrence commented on code in PR #831:
URL: https://github.com/apache/daffodil/pull/831#discussion_r950535870


##########
daffodil-tdml-processor/src/main/scala/org/apache/daffodil/tdml/processor/DaffodilTDMLDFDLProcessor.scala:
##########
@@ -139,18 +139,31 @@ final class TDMLDFDLProcessorFactory private (
     }
   }
 
+
   private def compileProcessor(
     schemaSource: DaffodilSchemaSource,
     useSerializedProcessor: Boolean,
     optRootName: Option[String],
     optRootNamespace: Option[String]): TDML.CompileResult = {
-    val pf = compiler.compileSource(schemaSource, optRootName, optRootNamespace)
-    val diags = pf.getDiagnostics
-    if (pf.isError) {
-      Left(diags)
+    //
+    // if the schemaSource is not an XSD file, then we assume it is
+    // a pre-compiled DFDL schema
+    //
+    if (schemaSource.isXSD)
+    {
+      val pf = compiler.compileSource(schemaSource, optRootName, optRootNamespace)
+      val diags = pf.getDiagnostics
+      if (pf.isError) {
+        Left(diags)
+      } else {
+        val res = this.generateProcessor(pf, useSerializedProcessor)
+        res
+      }
     } else {
-      val res = this.generateProcessor(pf, useSerializedProcessor)
-      res
+      val chan = Channels.newChannel(schemaSource.uriForLoading.toURL.openStream())
+      val dp = compiler.reload(chan)
+      val diags = dp.getDiagnostics
+      Right((diags, new DaffodilTDMLDFDLProcessor(dp)))

Review Comment:
   In general, I think it's a not a great idea to store precompiled binaries in a repository along with the schema/tests. Any update to Daffodil (including snapshots) are likely to not be compatible, so it's going to be pretty fragile. And any updates to the schema requires you to rebuild the precompiled binary--keeping those in sync sounds difficult and easy to forget.
   
   Ideally we would add a better caching mechanism than what we have right now (e.g. cache to a file instead of to memory). Or as I think you've suggested in the past, when we build the schema jar we also build a precompiled binary into the jar that the TDML runner could look for. That way we can rely on sbt to track when the schema changes and rebuilds the jar/precompiled binary as needed and you don't have to worry about storing one in the repo. That probably requires a special Daffodil plugin though or special build.sbt logic, so probably not a trivial amount of work.
   
   This feels fine as a temporary solution, but I hope we are able to deprecated it relatively soon and not make this standard practice.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org