You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by ji...@apache.org on 2020/11/30 15:51:16 UTC
[incubator-daffodil] 01/01: WIP: Add runtime2 backend with C code
generator
This is an automated email from the ASF dual-hosted git repository.
jinterrante pushed a commit to branch runtime2-2202
in repository https://gitbox.apache.org/repos/asf/incubator-daffodil.git
commit 0677a7514c8e5a02a5d451ae54361f126b5aecb8
Author: John Interrante <in...@research.ge.com>
AuthorDate: Sun Sep 20 11:42:14 2020 -0400
WIP: Add runtime2 backend with C code generator
Add a new C code generator runtime2 backend to Daffodil. Use a
runtime2-2202 branch, not the main branch, to invite collaboration on
building out code generator and shine light on any issues. Our goal
is to implement the smallest possible subset of DFDL in runtime2 as
described in the "Runtime 2 Design" table of
<https://cwiki.apache.org/confluence/display/DAFFODIL/WIP%3A+Daffodil+Runtime+2>.
At this point, code generator generates C code to parse and unparse
only 32-bit big-endian binary signed integers. However, you already
can run TDML tests and generate C code into an output directory with
daffodil commands like these,
```bash
$ cd daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/
$ daffodil test TestRuntime2.tdml
Creating DFDL Test Suite for TestRuntime2.tdml
[Pass] parse_int32
[Pass] unparse_int32
Total: 2, Pass: 2, Fail: 0, Not Found: 0
$ daffodil generate c -s TestRuntime2.dfdl.xsd outdir
$ ls -R outdir/
outdir/:
c
outdir/c:
daffodil_argp.c daffodil_argp.h daffodil_main.c generated_code.c generated_code.h infoset.c infoset.h stack.c stack.h xml_reader.c xml_reader.h xml_writer.c xml_writer.h
```
Note we will need to rebase the runtime2-2202 branch from time to time
to keep up with changes on the main branch. In order to avoid
problems pulling new changes into your checkout of the runtime2-2202
branch and seeing what was changed, you will want to use these
specific commands:
```bash
git switch runtime2-2202
git pull --rebase
git diff ORIG_HEAD..HEAD
```
You will need a C11 or C18 compiler ("zig cc", "gcc", "clang", or
"cc") and a tiny XML library ("libmxml.a" with header file "mxml.h").
You can install both of them on many systems using installable binary
packages or build and install the library from its source (the GitHub
repository "michaelrsweet/mxml").
DAFFODIL-2202
______________________________________________________________________
Questions & loose ends:
1. We want to generate C code that parses and unparses binary data as
securely as possible. Can we develop C functions that are more secure
than what the current code generator uses, such as a library of parser
combinator functions? That will be our goal as we build out the code
generator.
2. In CodeGeneratorState.scala's defineQNameInit method, how should we
handle multiple xmls=ns declarations?
3. How much do you want the "daffodil generate" command to build an
executable for you? Right now "daffodil generate c" only generates C
source files just like its name says. I'm not sure how many
developers will want to use the C-based CLI we've implemented in order
to run TDML tests. Would you want "daffodil generate c" to build an
executable that can convert data files to XML files (outdir/c/daffodil
parse <opts>) and back to data files (outdir/c/daffodil unparse
<opts>)? Or should "daffodil generate c" just give you the C source
files so you can take whatever you need and throw away whatever you
don't need like the CLI and XML reader/writer?
4. Should we make alternative runtime backends more pluggable by using
SPI? For now, I've taken the easiest way (using Class.forName with a
match expression) in Compiler.scala and TDMLRunner.scala which allows
daffodil-core to call runtime2 without having compile time
dependencies on any runtime2 class.
5. I've avoided using heap-allocated storage anywhere in the C source
files by using only statically allocated storage to keep the code
simple and avoid security problems, but we may need to change some
buffer sizes later (see char[9999] in infoset.c and mxml_node_t[100]
in xml_writer.c). Note that the MXML library does use heap-allocated
storage (it copies all the strings passed into it and our XML
reader/writer has to free MXML documents after using them), but we
need the XML reader/writer only for running TDML tests.
---
.github/workflows/main.yml | 45 ++-
.github/workflows/sonarcloud.yml | 33 ++-
build.sbt | 46 +++-
.../daffodil/generating/TestCLIGenerateC.scala | 198 ++++++++++++++
.../src/main/scala/org/apache/daffodil/Main.scala | 104 ++++++-
.../org/apache/daffodil/compiler/Compiler.scala | 32 ++-
.../org/apache/daffodil/dsom/ElementBase.scala | 32 +--
.../daffodil/grammar/ElementBaseGrammarMixin.scala | 9 +-
.../daffodil/grammar/ElementDeclGrammarMixin.scala | 5 +-
.../org/apache/daffodil/grammar/Grammar.scala | 6 +-
.../org/apache/daffodil/grammar/GrammarTerm.scala | 2 +-
.../org/apache/daffodil/grammar/Production.scala | 6 +-
.../grammar/primitives/ElementCombinator.scala | 48 ++--
.../primitives/PrimitivesBinaryNumber.scala | 5 +-
.../grammar/primitives/SequenceChild.scala | 11 +-
.../grammar/primitives/SequenceCombinator.scala | 14 +-
.../grammar/primitives/SpecifiedLength.scala | 1 -
.../resources/org/apache/daffodil/xsd/tdml.xsd | 1 +
.../resources/org/apache/daffodil/xsd/dafext.xsd | 8 +
.../apache/daffodil/api/DFDLParserUnparser.scala | 40 ++-
.../apache/daffodil/processors/DataProcessor.scala | 16 +-
daffodil-runtime2/src/main/resources/.clang-format | 22 ++
.../src/main/resources/c/daffodil_argp.c | 302 +++++++++++++++++++++
.../src/main/resources/c/daffodil_argp.h | 46 +++-
.../src/main/resources/c/daffodil_main.c | 137 ++++++++++
daffodil-runtime2/src/main/resources/c/infoset.c | 204 ++++++++++++++
daffodil-runtime2/src/main/resources/c/infoset.h | 139 ++++++++++
daffodil-runtime2/src/main/resources/c/stack.c | 85 ++++++
daffodil-runtime2/src/main/resources/c/stack.h | 63 +++++
.../src/main/resources/c/xml_reader.c | 210 ++++++++++++++
.../src/main/resources/c/xml_reader.h | 30 +-
.../src/main/resources/c/xml_writer.c | 118 ++++++++
.../src/main/resources/c/xml_writer.h | 28 +-
.../src/main/resources/examples/ex_int32.c | 260 ++++++++++++++++++
.../src/main/resources/examples/ex_int32.h | 30 +-
.../apache/daffodil/runtime2/CodeGenerator.scala | 179 ++++++++++++
.../daffodil/runtime2/Runtime2CodeGenerator.scala | 67 +++++
.../daffodil/runtime2/Runtime2DataProcessor.scala | 206 ++++++++++++++
.../BinaryIntegerKnownLengthCodeGenerator.scala | 91 +++++++
.../runtime2/generators/CodeGeneratorState.scala | 280 +++++++++++++++++++
...entParseAndUnspecifiedLengthCodeGenerator.scala | 55 ++++
.../generators/OrderedSequenceCodeGenerator.scala | 24 +-
.../runtime2/generators/SeqCompCodeGenerator.scala | 20 +-
.../runtime2/Runtime2TDMLDFDLProcessor.scala | 232 ++++++++++++++++
.../apache/daffodil/runtime2/TestRuntime2.dfdl.xsd | 50 ++++
.../org/apache/daffodil/runtime2/TestRuntime2.tdml | 67 +++++
.../org/apache/daffodil/runtime2/parse_int32 | Bin 0 -> 12 bytes
.../org/apache/daffodil/runtime2/unparse_int32 | 25 ++
.../daffodil/runtime2/TestCodeGenerator.scala | 144 ++++++++++
.../apache/daffodil/runtime2/TestRuntime2.scala | 24 +-
.../org/apache/daffodil/tdml/TDMLRunner.scala | 9 +-
project/Dependencies.scala | 6 +-
project/Rat.scala | 1 +
project/plugins.sbt | 2 +
54 files changed, 3614 insertions(+), 204 deletions(-)
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 12ae8c8..5fbd583 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -25,15 +25,20 @@ jobs:
matrix:
java_version: [ 8, 11, 15 ]
scala_version: [ 2.12.11 ]
- os: [ 'ubuntu-latest', 'windows-latest' ]
- env:
- SBT: sbt -J-Xms1024m -J-Xmx5120m -J-XX:ReservedCodeCacheSize=512m -J-XX:MaxMetaspaceSize=1024m ++${{ matrix.scala_version }} coverage
- SBTNOCOV: sbt -J-Xms1024m -J-Xmx5120m -J-XX:ReservedCodeCacheSize=512m -J-XX:MaxMetaspaceSize=1024m ++${{ matrix.scala_version }}
+ os: [ ubuntu-20.04, windows-latest ]
+ include:
+ - os: ubuntu-20.04
+ shell: bash
+ - os: windows-latest
+ shell: msys2 {0}
runs-on: ${{ matrix.os }}
defaults:
run:
- shell: bash
+ shell: ${{ matrix.shell }}
+ env:
+ SBT: sbt -J-Xms1024m -J-Xmx5120m -J-XX:ReservedCodeCacheSize=512m -J-XX:MaxMetaspaceSize=1024m ++${{ matrix.scala_version }} coverage
+ SBTNOCOV: sbt -J-Xms1024m -J-Xmx5120m -J-XX:ReservedCodeCacheSize=512m -J-XX:MaxMetaspaceSize=1024m ++${{ matrix.scala_version }}
steps:
@@ -41,14 +46,40 @@ jobs:
# Setup
############################################################
- - name: Checkout Repository
- uses: actions/checkout@v2.0.0
+ - name: Install Dependencies (Linux)
+ run: sudo apt-get install -y libmxml-dev
+ if: runner.os == 'Linux'
+
+ - name: Install Dependencies (Windows)
+ uses: msys2/setup-msys2@v2
+ if: runner.os == 'Windows'
+ with:
+ install: gcc libargp-devel make pkg-config
+ path-type: inherit
+ - name: Check out mxml source (Windows)
+ uses: actions/checkout@v2.0.0
+ if: runner.os == 'Windows'
+ with:
+ repository: michaelrsweet/mxml
+ path: mxml
+
+ - name: Install mxml library (Windows)
+ if: runner.os == 'Windows'
+ run: |
+ cd mxml
+ ./configure --prefix=/usr --disable-shared --disable-threads
+ make
+ make install
+
- name: Install Java
uses: actions/setup-java@v1
with:
java-version: ${{ matrix.java_version }}
+ - name: Checkout Repository
+ uses: actions/checkout@v2.0.0
+
############################################################
# Build & Package
############################################################
diff --git a/.github/workflows/sonarcloud.yml b/.github/workflows/sonarcloud.yml
index 03b4b48..c7d7b40 100644
--- a/.github/workflows/sonarcloud.yml
+++ b/.github/workflows/sonarcloud.yml
@@ -14,22 +14,40 @@
# limitations under the License.
name: SonarCloud Workflow
+
on: [push]
+
jobs:
- sonarCloudTrigger:
- if: github.repository == 'apache/incubator-daffodil'
- name: SonarCloud Trigger
- runs-on: ubuntu-latest
+ sonarcloud:
+ name: SonarCloud
strategy:
fail-fast: false
matrix:
- scala_version: [ '2.12.11']
+ java_version: [ 11 ]
+ scala_version: [ 2.12.11]
+ os: [ ubuntu-20.04 ]
+
+ runs-on: ${{ matrix.os }}
+ if: github.repository == 'apache/incubator-daffodil'
+ env:
+ SBT: sbt -J-Xms1024m -J-Xmx5120m -J-XX:ReservedCodeCacheSize=512m -J-XX:MaxMetaspaceSize=1024m ++${{ matrix.scala_version }} coverage
+ SBTNOCOV: sbt -J-Xms1024m -J-Xmx5120m -J-XX:ReservedCodeCacheSize=512m -J-XX:MaxMetaspaceSize=1024m ++${{ matrix.scala_version }}
+
steps:
############################################################
# Setup
############################################################
+ - name: Install Dependencies (Linux)
+ run: sudo apt-get install -y libmxml-dev
+ if: runner.os == 'Linux'
+
+ - name: Install Java
+ uses: actions/setup-java@v1
+ with:
+ java-version: ${{ matrix.java_version }}
+
- name: Checkout Repository
uses: actions/checkout@v2.0.0
@@ -38,10 +56,7 @@ jobs:
############################################################
- name: Compile
- run: $SBT compile test:compile it:compile
- shell: bash
- env:
- SBT: sbt -J-Xms1024m -J-Xmx5120m -J-XX:ReservedCodeCacheSize=512m -J-XX:MaxMetaspaceSize=1024m ++${{ matrix.scala_version }} coverage
+ run: $SBTNOCOV compile test:compile it:compile
- name: SonarCloud Scan
uses: sonarsource/sonarcloud-github-action@master
diff --git a/build.sbt b/build.sbt
index ed0eeac..b28704d 100644
--- a/build.sbt
+++ b/build.sbt
@@ -15,13 +15,17 @@
* limitations under the License.
*/
+import sbt.io.Path.flatRebase
+import sbtcc._
+import scala.collection.immutable.ListSet
+
lazy val genManaged = taskKey[Unit]("Generate managed sources and resources")
lazy val genProps = taskKey[Seq[File]]("Generate properties scala source")
lazy val genSchemas = taskKey[Seq[File]]("Generated DFDL schemas")
-lazy val daffodil = Project("daffodil", file(".")).configs(IntegrationTest)
+lazy val daffodil = project.in(file(".")).configs(IntegrationTest)
.enablePlugins(JavaUnidocPlugin, ScalaUnidocPlugin)
- .aggregate(macroLib, propgen, lib, io, runtime1, runtime1Unparser, core, japi, sapi, tdmlLib, tdmlProc, cli, udf, test, testIBM1, tutorials, testStdLayout)
+ .aggregate(macroLib, propgen, lib, io, runtime1, runtime1Unparser, runtime2, core, japi, sapi, tdmlLib, tdmlProc, cli, udf, test, testIBM1, tutorials, testStdLayout)
.settings(commonSettings, nopublish, ratSettings, unidocSettings)
lazy val macroLib = Project("daffodil-macro-lib", file("daffodil-macro-lib")).configs(IntegrationTest)
@@ -43,6 +47,34 @@ lazy val runtime1 = Project("daffodil-runtime1", file("daffodil-runtime1
.dependsOn(io, lib % "test->test", udf, macroLib % "compile-internal, test-internal")
.settings(commonSettings, usesMacros)
+val runtime2CFiles = Library("libruntime2.a")
+lazy val runtime2 = Project("daffodil-runtime2", file("daffodil-runtime2")).configs(IntegrationTest)
+ .enablePlugins(CcPlugin)
+ .dependsOn(core, core % "test->test", tdmlProc)
+ .settings(commonSettings)
+ .settings(publishArtifact in (Compile, packageDoc) := false)
+ .settings(
+ Compile / ccTargets := ListSet(runtime2CFiles),
+ Compile / cSources := Map(
+ runtime2CFiles -> (
+ ((Compile / resourceDirectory).value / "c" * GlobFilter("*.c")).get() ++
+ ((Compile / resourceDirectory).value / "examples" * GlobFilter("*.c")).get()
+ )
+ ),
+ Compile / cIncludeDirectories := Map(
+ runtime2CFiles -> Seq(
+ (Compile / resourceDirectory).value / "c",
+ (Compile / resourceDirectory).value / "examples"
+ )
+ ),
+ Compile / cFlags := (Compile / cFlags).value.withDefaultValue(Seq(
+ "-g",
+ "-Wall",
+ "-Wextra",
+ "-Wno-missing-field-initializers",
+ ))
+ )
+
lazy val runtime1Unparser = Project("daffodil-runtime1-unparser", file("daffodil-runtime1-unparser")).configs(IntegrationTest)
.dependsOn(runtime1, lib % "test->test", runtime1 % "test->test")
.settings(commonSettings)
@@ -68,9 +100,9 @@ lazy val tdmlProc = Project("daffodil-tdml-processor", file("daffodil-td
.settings(commonSettings)
lazy val cli = Project("daffodil-cli", file("daffodil-cli")).configs(IntegrationTest)
- .dependsOn(tdmlProc, sapi, japi, udf % "it->test") // causes sapi/japi to be pulled in to the helper zip/tar
+ .dependsOn(tdmlProc, runtime2, sapi, japi, udf % "it->test") // causes runtime2/sapi/japi to be pulled in to the helper zip/tar
.settings(commonSettings, nopublish)
- .settings(libraryDependencies ++= Dependencies.cli)
+ .settings(libraryDependencies ++= Dependencies.cli)
lazy val udf = Project("daffodil-udf", file("daffodil-udf")).configs(IntegrationTest)
.settings(commonSettings)
@@ -115,7 +147,7 @@ lazy val commonSettings = Seq(
"-Xfatal-warnings",
"-Xxml:-coalescing",
"-Xfuture"
-),
+ ),
// add scalac options that are version specific
scalacOptions ++= scalacCrossOptions(scalaVersion.value),
// Workaround issue that some options are valid for javac, not javadoc.
@@ -166,9 +198,9 @@ lazy val nopublish = Seq(
)
lazy val usesMacros = Seq(
- // copies classe and source files into the project that uses macros. Note
+ // copies classes and source files into the project that uses macros. Note
// that for packageBin, we only copy directories and class files--this
- // ignores files such a META-INFA/LICENSE and NOTICE that are duplicated and
+ // ignores files such a META-INF/LICENSE and NOTICE that are duplicated and
// would otherwise cause a conflict
mappings in (Compile, packageBin) ++= mappings.in(macroLib, Compile, packageBin).value.filter { case (f, _) => f.isDirectory || f.getPath.endsWith(".class") },
mappings in (Compile, packageSrc) ++= mappings.in(macroLib, Compile, packageSrc).value
diff --git a/daffodil-cli/src/it/scala/org/apache/daffodil/generating/TestCLIGenerateC.scala b/daffodil-cli/src/it/scala/org/apache/daffodil/generating/TestCLIGenerateC.scala
new file mode 100644
index 0000000..9b5c9b3
--- /dev/null
+++ b/daffodil-cli/src/it/scala/org/apache/daffodil/generating/TestCLIGenerateC.scala
@@ -0,0 +1,198 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.generating
+
+import org.junit.Test
+import org.junit.Before
+import org.junit.After
+import org.apache.daffodil.CLI.Util
+import net.sf.expectit.matcher.Matchers.contains
+import net.sf.expectit.matcher.Matchers.eof
+
+class TestCLIGenerateC {
+
+ val daffodil: String = Util.binPath
+ val outputDir: os.Path = os.pwd/"generate_tmp"
+ lazy val schemaFile: String = if (Util.isWindows) Util.cmdConvert(sf) else sf
+ val sf: String = Util.daffodilPath("daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/TestRuntime2.dfdl.xsd")
+
+ @Before def before(): Unit = {
+ os.remove.all(outputDir)
+ }
+
+ @After def after(): Unit = {
+ os.remove.all(outputDir)
+ }
+
+ @Test def test_CLI_Generate_schema(): Unit = {
+ val generateCmd = s"$daffodil generate c -s $schemaFile $outputDir"
+ val exitCmd = "exit"
+
+ val shell = Util.start("")
+ try {
+ shell.sendLine(generateCmd)
+ shell.sendLine(exitCmd)
+ shell.expect(eof())
+ } finally {
+ shell.close()
+ }
+
+ assert(os.exists(outputDir/"c"/"generated_code.c"))
+ }
+
+ @Test def test_CLI_Generate_noC_error(): Unit = {
+ val generateCmd = s"$daffodil generate -s $schemaFile $outputDir"
+ val exitCmd = "exit"
+
+ val shell = Util.start("", expectErr = true)
+ try {
+ shell.sendLine(generateCmd)
+ shell.expect(contains("Unknown option 's'"))
+ shell.sendLine(exitCmd)
+ shell.expect(eof())
+ } finally {
+ shell.close()
+ }
+ }
+
+ @Test def test_CLI_Generate_otherThanC_error(): Unit = {
+ val generateCmd = s"$daffodil generate vhld -s $schemaFile $outputDir"
+ val exitCmd = "exit"
+
+ val shell = Util.start("", expectErr = true)
+ try {
+ shell.sendLine(generateCmd)
+ shell.expect(contains("Excess arguments provided: 'vhld"))
+ shell.sendLine(exitCmd)
+ shell.expect(eof())
+ } finally {
+ shell.close()
+ }
+ }
+
+ @Test def test_CLI_Generate_noSchema_error(): Unit = {
+ val generateCmd = s"$daffodil generate c $outputDir"
+ val exitCmd = "exit"
+
+ val shell = Util.start("", expectErr = true)
+ try {
+ shell.sendLine(generateCmd)
+ shell.expect(contains("Required option 'schema' not found"))
+ shell.sendLine(exitCmd)
+ shell.expect(eof())
+ } finally {
+ shell.close()
+ }
+ }
+
+ @Test def test_CLI_Generate_twoSchema_error(): Unit = {
+ val generateCmd = s"$daffodil generate c -s $schemaFile -s $schemaFile $outputDir"
+ val exitCmd = "exit"
+
+ val shell = Util.start("", expectErr = true)
+ try {
+ shell.sendLine(generateCmd)
+ shell.expect(contains("you should provide exactly one argument"))
+ shell.sendLine(exitCmd)
+ shell.expect(eof())
+ } finally {
+ shell.close()
+ }
+ }
+
+ @Test def test_CLI_Generate_verbose(): Unit = {
+ val generateCmd = s"$daffodil -v generate c -s $schemaFile $outputDir"
+ val exitCmd = "exit"
+
+ val shell = Util.start("", expectErr = true)
+ try {
+ shell.sendLine(generateCmd)
+ shell.expect(contains("[info] Time (compiling)"))
+ shell.expect(contains("[info] Time (generating)"))
+ shell.sendLine(exitCmd)
+ shell.expect(eof())
+ } finally {
+ shell.close()
+ }
+
+ assert(os.exists(outputDir/"c"/"generated_code.c"))
+ }
+
+ @Test def test_CLI_Generate_root(): Unit = {
+ val generateCmd = s"$daffodil generate c -s $schemaFile -r {http://example.com}ex_int32 $outputDir"
+ val exitCmd = "exit"
+
+ val shell = Util.start("")
+ try {
+ shell.sendLine(generateCmd)
+ shell.sendLine(exitCmd)
+ shell.expect(eof())
+ } finally {
+ shell.close()
+ }
+
+ assert(os.exists(outputDir/"c"/"generated_code.c"))
+ }
+
+ @Test def test_CLI_Generate_root_error(): Unit = {
+ val generateCmd = s"$daffodil generate c -s $schemaFile -r {ex}ex_int32 $outputDir"
+ val exitCmd = "exit"
+
+ val shell = Util.start("", expectErr = true)
+ try {
+ shell.sendLine(generateCmd)
+ shell.expect(contains("Schema Definition Error"))
+ shell.expect(contains("No global element found for {ex}ex_int32"))
+ shell.sendLine(exitCmd)
+ shell.expect(eof())
+ } finally {
+ shell.close()
+ }
+ }
+
+ @Test def test_CLI_Generate_namespaceNoRoot_error(): Unit = {
+ val generateCmd = s"$daffodil generate c -s $schemaFile -r {http://example.com} $outputDir"
+ val exitCmd = "exit"
+
+ val shell = Util.start("", expectErr = true)
+ try {
+ shell.sendLine(generateCmd)
+ shell.expect(contains("'{http://example.com}' - wrong arguments format"))
+ shell.sendLine(exitCmd)
+ shell.expect(eof())
+ } finally {
+ shell.close()
+ }
+ }
+
+ @Test def test_CLI_Generate_tunable(): Unit = {
+ val generateCmd = s"$daffodil generate c -s $schemaFile -T parseUnparsePolicy=parseOnly $outputDir"
+ val exitCmd = "exit"
+
+ val shell = Util.start("")
+ try {
+ shell.sendLine(generateCmd)
+ shell.sendLine(exitCmd)
+ shell.expect(eof())
+ } finally {
+ shell.close()
+ }
+
+ assert(os.exists(outputDir/"c"/"generated_code.c"))
+ }
+}
diff --git a/daffodil-cli/src/main/scala/org/apache/daffodil/Main.scala b/daffodil-cli/src/main/scala/org/apache/daffodil/Main.scala
index 5b6f18e..78b0c8f 100644
--- a/daffodil-cli/src/main/scala/org/apache/daffodil/Main.scala
+++ b/daffodil-cli/src/main/scala/org/apache/daffodil/Main.scala
@@ -38,7 +38,6 @@ import scala.concurrent.duration.Duration
import scala.language.reflectiveCalls
import scala.xml.Node
import scala.xml.SAXParseException
-
import javax.xml.parsers.DocumentBuilderFactory
import javax.xml.parsers.SAXParserFactory
import javax.xml.transform.TransformerFactory
@@ -541,11 +540,48 @@ class CLIConf(arguments: Array[String]) extends scallop.ScallopConf(arguments)
val info = tally(descr = "increment test result information output level, one level for each -i")
}
+ // Generate Subcommand Options
+ val generate = new scallop.Subcommand("generate") {
+ descr("generate <language> code from a DFDL schema")
+
+ banner("""|Usage: daffodil [GLOBAL_OPTS] generate <language> [SUBCOMMAND_OPTS]
+ |""".stripMargin)
+ shortSubcommandsHelp()
+ footer("""|
+ |Run 'daffodil generate <language> --help' for subcommand specific options""".stripMargin)
+
+ val c = new scallop.Subcommand("c") {
+ banner("""|Usage: daffodil generate c -s <schema> [-r [{namespace}]<root>]
+ | [-c <file>] [outputDir]
+ |
+ |Generate C code from a DFDL schema to parse or unparse data
+ |
+ |Generate Options:""".stripMargin)
+
+ descr("generate C code from a DFDL schema")
+ helpWidth(76)
+
+ val language = "c"
+ val schema = opt[URI]("schema", required = true, argName = "file", descr = "the annotated DFDL schema to use to generate source code.")
+ val rootNS = opt[RefQName]("root", argName = "node", descr = "the root element of the XML file to use. An optional namespace may be provided. This needs to be one of the top-level elements of the DFDL schema defined with --schema. Requires --schema. If not supplied uses the first element of the first schema")
+ val tunables = props[String]('T', keyName = "tunable", valueName = "value", descr = "daffodil tunable to be used when compiling schema.")
+ val config = opt[String](short = 'c', argName = "file", descr = "path to file containing configuration items.")
+ val outputDir = trailArg[String](required = false, descr = "output directory in which to generate source code. If not specified, uses current directory.")
+
+ validateOpt(schema) {
+ case None => Left("No schemas specified using the --schema option")
+ case _ => Right(Unit)
+ }
+ }
+ addSubcommand(c)
+ }
+
addSubcommand(parse)
addSubcommand(performance)
addSubcommand(unparse)
addSubcommand(save)
addSubcommand(test)
+ addSubcommand(generate)
validateOpt(trace, debug) {
case (Some(true), Some(_)) => Left("Only one of --trace and --debug may be defined")
@@ -700,7 +736,7 @@ object Main extends Logging {
def createProcessorFromSchema(schema: URI, rootNS: Option[RefQName], path: Option[String],
tunables: Map[String, String],
- mode: ValidationMode.Type) = {
+ mode: ValidationMode.Type): Option[DFDL.DataProcessor] = {
val compiler = {
val c = Compiler().withTunables(tunables)
rootNS match {
@@ -731,6 +767,31 @@ object Main extends Logging {
pf
}
+ def createGeneratorFromSchema(schema: URI, rootNS: Option[RefQName], tunables: Map[String, String],
+ language: String): Option[DFDL.CodeGenerator] = {
+ val compiler = {
+ val c = Compiler().withTunables(tunables)
+ rootNS match {
+ case None => c
+ case Some(RefQName(_, root, ns)) => c.withDistinguishedRootNode(root, ns.toStringOrNullIfNoNS)
+ }
+ }
+
+ val schemaSource = URISchemaSource(schema)
+ val cg = Timer.getResult("compiling", {
+ val processorFactory = compiler.compileSource(schemaSource)
+ if (!processorFactory.isError) {
+ val generator = processorFactory.forLanguage(language)
+ displayDiagnostics(generator)
+ Some(generator)
+ } else {
+ displayDiagnostics(processorFactory)
+ None
+ }
+ })
+ cg
+ }
+
// write blobs to $PWD/daffodil-blobs/*.bin
val blobDir = Paths.get(System.getProperty("user.dir"), "daffodil-blobs")
val blobSuffix = ".bin"
@@ -1433,11 +1494,42 @@ object Main extends Logging {
0
}
- case _ => {
- // This should never happen, this is caught by validation
- Assert.impossible()
- // 1
+ case Some(conf.generate) => {
+ conf.subcommands match {
+ case List(conf.generate, conf.generate.c) => {
+ val generateOpts = conf.generate.c
+
+ // Read any config file and any tunables given as arguments
+ val cfgFileNode = generateOpts.config.toOption match {
+ case None => None
+ case Some(pathToConfig) => Some(this.loadConfigurationFile(pathToConfig))
+ }
+ val tunables = retrieveTunables(generateOpts.tunables, cfgFileNode)
+
+ // Create a CodeGenerator from the DFDL schema
+ val generator = createGeneratorFromSchema(generateOpts.schema(), generateOpts.rootNS.toOption,
+ tunables, generateOpts.language)
+
+ // Ask the CodeGenerator to generate source code from the DFDL schema
+ val rootNS = generateOpts.rootNS.toOption
+ val outputDir = generateOpts.outputDir.toOption.getOrElse(".")
+ val rc = generator match {
+ case Some(generator) => {
+ Timer.getResult("generating", generator.generateCode(rootNS, outputDir))
+ displayDiagnostics(generator)
+ if (generator.isError) 1 else 0
+ }
+ case None => 1
+ }
+ rc
+ }
+ // Required to avoid "match may not be exhaustive", but should never happen
+ case _ => Assert.impossible()
+ }
}
+
+ // Required to avoid "match may not be exhaustive", but should never happen
+ case _ => Assert.impossible()
}
ret
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/compiler/Compiler.scala b/daffodil-core/src/main/scala/org/apache/daffodil/compiler/Compiler.scala
index 3c74654..714ac12 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/compiler/Compiler.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/compiler/Compiler.scala
@@ -26,6 +26,7 @@ import java.util.zip.GZIPInputStream
import java.util.zip.ZipException
import scala.collection.immutable.Queue
+import scala.util.Try
import scala.xml.Node
import org.apache.daffodil.api.DFDL
import org.apache.daffodil.api.DaffodilSchemaSource
@@ -108,6 +109,27 @@ final class ProcessorFactory private(
override def onPath(xpath: String) = sset.onPath(xpath)
+ override def forLanguage(language: String): DFDL.CodeGenerator = {
+ Assert.usage(!isError)
+
+ // Do a poor man's pluggable code generator implementation - we can replace
+ // it after we observe how the validator SPI evolves and wait for our
+ // requirements to become clearer
+ val className = language match {
+ case "c" => "org.apache.daffodil.runtime2.CodeGenerator"
+ case _ => s"code generator; source language $language is not supported"
+ }
+ import scala.language.existentials // Needed to make next line compile
+ val clazz = Try(Class.forName(className))
+ val constructor = clazz.map { _.getDeclaredConstructor(sset.root.getClass) }
+ val tryInstance = constructor.map { _.newInstance(sset.root).asInstanceOf[DFDL.CodeGenerator] }
+ val codeGenerator = tryInstance.recover {
+ case ex => throw new InvalidParserException("Error creating " + className, ex)
+ }.get
+
+ codeGenerator
+ }
+
override def isError = sset.isError
@deprecated("Use arguments to Compiler.compileSource or compileFile.", "2.6.0")
@@ -163,7 +185,7 @@ class Compiler private (var validateDFDLSchemas: Boolean,
}
@deprecated("Use constructor argument.", "2.6.0")
- def setValidateDFDLSchemas(value: Boolean) = validateDFDLSchemas = value
+ def setValidateDFDLSchemas(value: Boolean): Unit = validateDFDLSchemas = value
def withValidateDFDLSchemas(value: Boolean) = copy(validateDFDLSchemas = value)
@@ -279,10 +301,10 @@ class Compiler private (var validateDFDLSchemas: Boolean,
//
case cnf: ClassNotFoundException => {
val cpString =
- if (Misc.classPath.length == 0) " empty."
+ if (Misc.classPath.isEmpty) " empty."
else ":\n" + Misc.classPath.mkString("\n\t")
val fmtMsg = "%s\nThe class may not exist in this Java JVM version (%s), or it is missing from the classpath which is%s"
- val msg = fmtMsg.format(cnf.getMessage(), scala.util.Properties.javaVersion, cpString)
+ val msg = fmtMsg.format(cnf.getMessage, scala.util.Properties.javaVersion, cpString)
throw new InvalidParserException(msg, cnf)
}
}
@@ -329,10 +351,10 @@ class Compiler private (var validateDFDLSchemas: Boolean,
val err = pf.isError
val diags = pf.getDiagnostics // might be warnings even if not isError
if (err) {
- Assert.invariant(diags.length > 0)
+ Assert.invariant(diags.nonEmpty)
log(LogLevel.Compile, "Compilation (ProcessorFactory) produced %d errors/warnings.", diags.length)
} else {
- if (diags.length > 0) {
+ if (diags.nonEmpty) {
log(LogLevel.Compile, "Compilation (ProcessorFactory) produced %d warnings.", diags.length)
} else {
log(LogLevel.Compile, "ProcessorFactory completed with no errors.")
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/dsom/ElementBase.scala b/daffodil-core/src/main/scala/org/apache/daffodil/dsom/ElementBase.scala
index e1435d9..965dfd9 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/dsom/ElementBase.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/dsom/ElementBase.scala
@@ -32,7 +32,7 @@ import org.apache.daffodil.dpath.NodeInfo.PrimType
import org.apache.daffodil.dpath.InvalidPrimitiveDataException
import org.apache.daffodil.exceptions.Assert
import org.apache.daffodil.api.WarnID
-import java.lang.{Integer => JInt}
+import java.lang.{ Integer => JInt }
import org.apache.daffodil.dsom.walker.ElementBaseView
import org.apache.daffodil.infoset.DataValue
@@ -122,8 +122,6 @@ trait ElementBase
*/
def typeDef: TypeBase
- private lazy val optRepValueSet = optSimpleType.flatMap(_.optRepValueSet)
-
/**
* The DPathElementInfo objects referenced within an IVC
* that calls dfdl:contentLength( thingy )
@@ -441,12 +439,12 @@ trait ElementBase
}
final lazy val isParentUnorderedSequence: Boolean = {
- optLexicalParent.map { lp =>
+ optLexicalParent.exists { lp =>
lp match {
case s: SequenceTermBase if !s.isOrdered => true
case _ => false
}
- }.getOrElse(false)
+ }
}
private def getImplicitAlignmentInBits(thePrimType: PrimType, theRepresentation: Representation): Int = {
@@ -793,16 +791,16 @@ trait ElementBase
import org.apache.daffodil.dsom.FacetTypes._
- private lazy val hasPattern: Boolean = typeDef.optRestriction.map { _.hasPattern }.getOrElse(false)
- private lazy val hasEnumeration: Boolean = typeDef.optRestriction.map { _.hasEnumeration }.getOrElse(false)
- protected lazy val hasMinLength = typeDef.optRestriction.map { _.hasMinLength }.getOrElse(false)
- protected lazy val hasMaxLength = typeDef.optRestriction.map { _.hasMaxLength }.getOrElse(false)
- private lazy val hasMinInclusive = typeDef.optRestriction.map { _.hasMinInclusive }.getOrElse(false)
- private lazy val hasMaxInclusive = typeDef.optRestriction.map { _.hasMaxInclusive }.getOrElse(false)
- private lazy val hasMinExclusive = typeDef.optRestriction.map { _.hasMinExclusive }.getOrElse(false)
- private lazy val hasMaxExclusive = typeDef.optRestriction.map { _.hasMaxExclusive }.getOrElse(false)
- private lazy val hasTotalDigits = typeDef.optRestriction.map { _.hasTotalDigits }.getOrElse(false)
- private lazy val hasFractionDigits = typeDef.optRestriction.map { _.hasFractionDigits }.getOrElse(false)
+ private lazy val hasPattern: Boolean = typeDef.optRestriction.exists(_.hasPattern)
+ private lazy val hasEnumeration: Boolean = typeDef.optRestriction.exists(_.hasEnumeration)
+ protected lazy val hasMinLength = typeDef.optRestriction.exists(_.hasMinLength)
+ protected lazy val hasMaxLength = typeDef.optRestriction.exists(_.hasMaxLength)
+ private lazy val hasMinInclusive = typeDef.optRestriction.exists(_.hasMinInclusive)
+ private lazy val hasMaxInclusive = typeDef.optRestriction.exists(_.hasMaxInclusive)
+ private lazy val hasMinExclusive = typeDef.optRestriction.exists(_.hasMinExclusive)
+ private lazy val hasMaxExclusive = typeDef.optRestriction.exists(_.hasMaxExclusive)
+ private lazy val hasTotalDigits = typeDef.optRestriction.exists(_.hasTotalDigits)
+ private lazy val hasFractionDigits = typeDef.optRestriction.exists(_.hasFractionDigits)
final lazy val patternValues: Seq[FacetValueR] = {
Assert.invariant(hasPattern)
@@ -843,7 +841,7 @@ trait ElementBase
// we convert to the number type.
//
// This means we cannot check and SDE here on incorrect simple type.
- return (zeroBD, unbBD)
+ (zeroBD, unbBD)
}
case st: SimpleTypeDefBase if st.optRepTypeElement.isDefined => (st.optRepTypeElement.get.minLength, st.optRepTypeElement.get.maxLength)
case st: SimpleTypeDefBase if st.optRestriction.isDefined => {
@@ -1072,8 +1070,6 @@ trait ElementBase
}
}
- private def gcd(a: Int, b: Int): Int = if (b == 0) a else gcd(b, a % b)
-
/**
* Changed to a warning - DFDL WG decided to make this check optional, but it
* is still useful as a warning.
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementBaseGrammarMixin.scala b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementBaseGrammarMixin.scala
index 14be1d7..61da856 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementBaseGrammarMixin.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementBaseGrammarMixin.scala
@@ -18,14 +18,11 @@
package org.apache.daffodil.grammar
import java.lang.{ Long => JLong }
-import scala.Boolean
-import scala.Long
import org.apache.daffodil.api.WarnID
import org.apache.daffodil.dpath.NodeInfo
import org.apache.daffodil.dpath.NodeInfo.PrimType
import org.apache.daffodil.dsom.PrefixLengthQuasiElementDecl
import org.apache.daffodil.dsom.ElementBase
-import org.apache.daffodil.dsom.ExpressionCompilers
import org.apache.daffodil.dsom.InitiatedTerminatedMixin
import org.apache.daffodil.dsom.TunableLimitExceededError
import org.apache.daffodil.exceptions.Assert
@@ -1016,7 +1013,7 @@ trait ElementBaseGrammarMixin
}
private def withDelimiterStack(body: => Gram) = {
- if (hasDelimiters || immediatelyEnclosingModelGroup.map(_.hasDelimiters).getOrElse(false)) DelimiterStackCombinatorElement(this, body)
+ if (hasDelimiters || immediatelyEnclosingModelGroup.exists(_.hasDelimiters)) DelimiterStackCombinatorElement(this, body)
else body
}
@@ -1185,7 +1182,7 @@ trait ElementBaseGrammarMixin
val exprNamespaces = exprProp.location.namespaces
val qn = GlobalQName(Some("daf"), "outputValueCalc", XMLUtils.dafintURI)
val expr = ExpressionCompilers.AnyRef.compileExpression(
- qn, primType, exprText, exprNamespaces, dpathCompileInfo, false, self, dpathCompileInfo)
+ qn, primType, exprText, exprNamespaces, dpathCompileInfo, isEvaluatedAbove = false, self, dpathCompileInfo)
expr
}
@@ -1299,7 +1296,7 @@ trait ElementBaseGrammarMixin
val len = optLengthConstant.get
val maxLengthLong = maxLength.longValueExact
val minLengthLong = minLength.longValueExact
- def warn(m: String, value: Long) = SDW(WarnID.FacetExplicitLengthOutOfRange, "Explicit dfdl:length of %s is out of range for facet %sLength='%s'.", len, m, value)
+ def warn(m: String, value: Long): Unit = SDW(WarnID.FacetExplicitLengthOutOfRange, "Explicit dfdl:length of %s is out of range for facet %sLength='%s'.", len, m, value)
if (maxLengthLong != -1 && len > maxLengthLong) warn("max", maxLengthLong)
Assert.invariant(minLengthLong >= 0)
if (minLengthLong > 0 && len < minLengthLong) warn("min", minLengthLong)
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
index f0bc051..c294f68 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
@@ -17,8 +17,8 @@
package org.apache.daffodil.grammar
-import org.apache.daffodil.grammar.primitives.UnicodeByteOrderMark
import org.apache.daffodil.dsom.Root
+import org.apache.daffodil.grammar.primitives.UnicodeByteOrderMark
trait RootGrammarMixin
extends LocalElementGrammarMixin // can be repeating if not root
@@ -29,6 +29,5 @@ trait RootGrammarMixin
UnicodeByteOrderMark(this) ~ documentElement
}
- private def documentElement = enclosedElement
-
+ def documentElement = enclosedElement
}
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/Grammar.scala b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/Grammar.scala
index 4941037..9b75e5f 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/Grammar.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/Grammar.scala
@@ -41,7 +41,7 @@ abstract class BinaryGram(context: SchemaComponent, childrenArg: Seq[Gram]) exte
protected def open: String
protected def close: String
- protected final lazy val children = {
+ lazy val children = {
val c = childrenArg
c
}
@@ -87,7 +87,7 @@ class SeqComp private (context: SchemaComponent, children: Seq[Gram]) extends Bi
final override lazy val parser = {
if (parserChildren.isEmpty) new NadaParser(context.runtimeData)
- else if (parserChildren.length == 1) parserChildren(0)
+ else if (parserChildren.length == 1) parserChildren.head
else new SeqCompParser(context.runtimeData, parserChildren.toVector)
}
@@ -105,7 +105,7 @@ class SeqComp private (context: SchemaComponent, children: Seq[Gram]) extends Bi
final override lazy val unparser = {
if (unparserChildren.isEmpty) new NadaUnparser(context.runtimeData)
- else if (unparserChildren.length == 1) unparserChildren(0)
+ else if (unparserChildren.length == 1) unparserChildren.head
else new SeqCompUnparser(context.runtimeData, unparserChildren.toVector)
}
}
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/GrammarTerm.scala b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/GrammarTerm.scala
index db042a9..cc464bc 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/GrammarTerm.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/GrammarTerm.scala
@@ -65,7 +65,7 @@ abstract class Gram(contextArg: SchemaComponent)
val forWhat: ParserOrUnparser = BothParserAndUnparser
- final val context: SchemaComponent = contextArg
+ val context: SchemaComponent = contextArg
def term = context.asInstanceOf[Term]
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/Production.scala b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/Production.scala
index 40a4580..d10ef4e 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/Production.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/Production.scala
@@ -41,7 +41,11 @@ import org.apache.daffodil.util.LogLevel
* Prod objects are not required. They essentially provide some useful debug capability
* because a grammar term object will display as it's name, not as some anonymous object.
*/
-final class Prod(nameArg: String, val sc: SchemaComponent, guard: Boolean, gramArg: => Gram, override val forWhat: ParserOrUnparser)
+final class Prod(nameArg: String,
+ val sc: SchemaComponent,
+ val guard: Boolean,
+ gramArg: => Gram,
+ override val forWhat: ParserOrUnparser)
extends NamedGram(sc) {
final override def deref = gram
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/ElementCombinator.scala b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/ElementCombinator.scala
index fd50d50..bf24043 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/ElementCombinator.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/ElementCombinator.scala
@@ -23,13 +23,13 @@ import org.apache.daffodil.exceptions.Assert
import org.apache.daffodil.grammar.Gram
import org.apache.daffodil.grammar.NamedGram
import org.apache.daffodil.grammar.Terminal
-import org.apache.daffodil.processors.parsers.ElementParser
-import org.apache.daffodil.processors.parsers.ElementParserInputValueCalc
-import org.apache.daffodil.processors.parsers.Parser
import org.apache.daffodil.processors.parsers.CaptureEndOfContentLengthParser
import org.apache.daffodil.processors.parsers.CaptureEndOfValueLengthParser
import org.apache.daffodil.processors.parsers.CaptureStartOfContentLengthParser
import org.apache.daffodil.processors.parsers.CaptureStartOfValueLengthParser
+import org.apache.daffodil.processors.parsers.ElementParser
+import org.apache.daffodil.processors.parsers.ElementParserInputValueCalc
+import org.apache.daffodil.processors.parsers.Parser
import org.apache.daffodil.processors.unparsers.CaptureEndOfContentLengthUnparser
import org.apache.daffodil.processors.unparsers.CaptureEndOfValueLengthUnparser
import org.apache.daffodil.processors.unparsers.CaptureStartOfContentLengthUnparser
@@ -41,10 +41,10 @@ import org.apache.daffodil.processors.unparsers.ElementUnparserInputValueCalc
import org.apache.daffodil.processors.unparsers.ElementUnspecifiedLengthUnparser
import org.apache.daffodil.processors.unparsers.ElementUnusedUnparser
import org.apache.daffodil.processors.unparsers.LeftCenteredPaddingUnparser
-import org.apache.daffodil.processors.unparsers.SimpleTypeRetryUnparser
import org.apache.daffodil.processors.unparsers.OnlyPaddingUnparser
import org.apache.daffodil.processors.unparsers.RightCenteredPaddingUnparser
import org.apache.daffodil.processors.unparsers.RightFillUnparser
+import org.apache.daffodil.processors.unparsers.SimpleTypeRetryUnparser
import org.apache.daffodil.processors.unparsers.Unparser
import org.apache.daffodil.schema.annotation.props.gen.LengthKind
import org.apache.daffodil.schema.annotation.props.gen.Representation
@@ -67,18 +67,18 @@ import org.apache.daffodil.grammar.EmptyGram
* is going to make it worse.
*/
class ElementCombinator(
- context: ElementBase,
- eBeforeContent: Gram,
- eValue: Gram,
- eAfterValue: Gram,
- repTypeElementGram: Gram = EmptyGram
- )
+ override val context: ElementBase,
+ val eBeforeContent: Gram,
+ val eValue: Gram,
+ val eAfterValue: Gram,
+ val repTypeElementGram: Gram = EmptyGram
+)
extends NamedGram(context)
with Padded {
override def toString = subComb.toString()
- private lazy val subComb = new ElementParseAndUnspecifiedLength(context, eBeforeContent,
+ lazy val subComb = new ElementParseAndUnspecifiedLength(context, eBeforeContent,
eValue, eAfterValue, repTypeElementGram)
override lazy val parser: Parser = {
@@ -105,7 +105,7 @@ class ElementCombinator(
else Maybe(eAfterValue.unparser)
private lazy val eReptypeUnparser: Maybe[Unparser] = repTypeElementGram.maybeUnparser
-
+
override lazy val unparser: Unparser = {
if (context.isOutputValueCalc) {
new ElementOVCSpecifiedLengthUnparser(
@@ -133,7 +133,6 @@ class ElementCombinator(
subComb.unparser
}
}
-
}
case class ElementUnused(ctxt: ElementBase)
@@ -282,8 +281,11 @@ case class CaptureValueLengthEnd(ctxt: ElementBase)
new NadaUnparser(ctxt.erd)
}
-
-class ElementParseAndUnspecifiedLength(context: ElementBase, eBeforeGram: Gram, eGram: Gram, eAfterGram: Gram, repTypeElementGram: Gram)
+class ElementParseAndUnspecifiedLength(override val context: ElementBase,
+ val eBeforeGram: Gram,
+ val eGram: Gram,
+ val eAfterGram: Gram,
+ val repTypeElementGram: Gram)
extends ElementCombinatorBase(context, eBeforeGram, eGram, eAfterGram, repTypeElementGram) {
lazy val parser: Parser =
@@ -330,10 +332,10 @@ class ElementParseAndUnspecifiedLength(context: ElementBase, eBeforeGram: Gram,
}
}
-abstract class ElementCombinatorBase(context: ElementBase, eGramBefore: Gram, eGram: Gram, eGramAfter: Gram, repTypeElementGram:Gram)
+abstract class ElementCombinatorBase(context: ElementBase, eGramBefore: Gram, eGram: Gram, eGramAfter: Gram, repTypeElementGram: Gram)
extends NamedGram(context) {
- override def toString() = "<element name='" + name + "'>" + eGram.toString() + "</element>"
+ override def toString: String = "<element name='" + name + "'>" + eGram.toString + "</element>"
// The order of things matters in some cases, so to be consistent we'll always use the
// same order even when it doesn't matter
@@ -352,10 +354,10 @@ abstract class ElementCombinatorBase(context: ElementBase, eGramBefore: Gram, eG
lazy val patDiscrim = {
val pd = context.discriminatorStatements.filter(_.testKind == TestKind.Pattern)
Assert.invariant(pd.size <= 1)
- if (pd.size == 0) {
+ if (pd.isEmpty) {
Maybe.Nope
} else {
- pd(0).gram(context).maybeParser
+ pd.head.gram(context).maybeParser
}
}
lazy val patAssert = context.assertStatements.filter(_.testKind == TestKind.Pattern).map(_.gram(context).parser).toArray
@@ -363,10 +365,10 @@ abstract class ElementCombinatorBase(context: ElementBase, eGramBefore: Gram, eG
lazy val testDiscrim = {
val td = context.discriminatorStatements.filter(_.testKind == TestKind.Expression)
Assert.invariant(td.size <= 1)
- if (td.size == 0) {
+ if (td.isEmpty) {
Maybe.Nope
} else {
- td(0).gram(context).maybeParser
+ td.head.gram(context).maybeParser
}
}
lazy val testAssert = context.assertStatements.filter(_.testKind == TestKind.Expression).map(_.gram(context).parser).toArray
@@ -376,7 +378,7 @@ abstract class ElementCombinatorBase(context: ElementBase, eGramBefore: Gram, eG
lazy val eParser: Maybe[Parser] = eGram.maybeParser
lazy val eAfterParser: Maybe[Parser] = eGramAfter.maybeParser
-
+
lazy val eRepTypeParser: Maybe[Parser] = repTypeElementGram.maybeParser
def parser: Parser
@@ -388,7 +390,7 @@ abstract class ElementCombinatorBase(context: ElementBase, eGramBefore: Gram, eG
lazy val eUnparser: Maybe[Unparser] = eGram.maybeUnparser
lazy val eAfterUnparser: Maybe[Unparser] = eGramAfter.maybeUnparser
-
+
lazy val eRepTypeUnparser: Maybe[Unparser] = repTypeElementGram.maybeUnparser
def unparser: Unparser
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/PrimitivesBinaryNumber.scala b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/PrimitivesBinaryNumber.scala
index 3e1423c..0939725 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/PrimitivesBinaryNumber.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/PrimitivesBinaryNumber.scala
@@ -47,13 +47,16 @@ class BinaryIntegerRuntimeLength(val e: ElementBase, signed: Boolean) extends Te
override lazy val unparser: Unparser = new BinaryIntegerRuntimeLengthUnparser(e.elementRuntimeData, signed, e.lengthEv, e.lengthUnits)
}
-class BinaryIntegerKnownLength(val e: ElementBase, signed: Boolean, lengthInBits: Long) extends Terminal(e, true) {
+class BinaryIntegerKnownLength(val e: ElementBase,
+ val signed: Boolean,
+ val lengthInBits: Long) extends Terminal(e, true) {
override lazy val parser = {
new BinaryIntegerKnownLengthParser(e.elementRuntimeData, signed, lengthInBits.toInt)
}
override lazy val unparser: Unparser = new BinaryIntegerKnownLengthUnparser(e.elementRuntimeData, signed, lengthInBits.toInt)
+
}
class BinaryIntegerPrefixedLength(val e: ElementBase, signed: Boolean) extends Terminal(e, true) {
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SequenceChild.scala b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SequenceChild.scala
index cf60870..ea6ed3c 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SequenceChild.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SequenceChild.scala
@@ -17,17 +17,16 @@
package org.apache.daffodil.grammar.primitives
-import org.apache.daffodil.grammar._
-import org.apache.daffodil.dsom._
-import org.apache.daffodil.processors.unparsers._
-import org.apache.daffodil.processors.parsers._
import org.apache.daffodil.dpath.NodeInfo
+import org.apache.daffodil.dsom._
import org.apache.daffodil.exceptions.Assert
-import org.apache.daffodil.schema.annotation.props.gen.OccursCountKind
+import org.apache.daffodil.grammar._
+import org.apache.daffodil.processors.parsers._
+import org.apache.daffodil.processors.unparsers._
import org.apache.daffodil.schema.annotation.props.SeparatorSuppressionPolicy
import org.apache.daffodil.schema.annotation.props.gen.LengthKind
+import org.apache.daffodil.schema.annotation.props.gen.OccursCountKind
import org.apache.daffodil.schema.annotation.props.gen.Representation
-import org.apache.daffodil.processors.parsers._
/**
* A SequenceChild is exactly that, a child Term of a Sequence
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SequenceCombinator.scala b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SequenceCombinator.scala
index f6f9185..2a4b035 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SequenceCombinator.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SequenceCombinator.scala
@@ -17,12 +17,12 @@
package org.apache.daffodil.grammar.primitives
-import org.apache.daffodil.grammar.Terminal
-import org.apache.daffodil.grammar.Gram
import org.apache.daffodil.dsom._
+import org.apache.daffodil.grammar.Gram
+import org.apache.daffodil.grammar.Terminal
import org.apache.daffodil.processors.parsers._
-import org.apache.daffodil.schema.annotation.props.SeparatorSuppressionPolicy
import org.apache.daffodil.processors.unparsers._
+import org.apache.daffodil.schema.annotation.props.SeparatorSuppressionPolicy
import org.apache.daffodil.util.Misc
import org.apache.daffodil.util.Maybe
import org.apache.daffodil.util.MaybeInt
@@ -31,14 +31,14 @@ import org.apache.daffodil.util.MaybeInt
* Base class for all kinds of sequences.
*/
abstract class SequenceCombinator(sq: SequenceTermBase, sequenceChildren: Seq[SequenceChild])
- extends Terminal(sq, sequenceChildren.length > 0) {
+ extends Terminal(sq, sequenceChildren.nonEmpty) {
/**
* Shorthand for getting the sequence runtime data.
*/
val srd = sq.sequenceRuntimeData
- override def toString() =
+ override def toString: String =
"<" + Misc.getNameFromClass(this) + ">" +
sequenceChildren.map { _.toString() }.mkString +
"</" + Misc.getNameFromClass(this) + ">"
@@ -70,7 +70,7 @@ class OrderedSequence(sq: SequenceTermBase, sequenceChildrenArg: Seq[SequenceChi
// the mta parser differently to avoid this
private lazy val sepUnparser = sepGram.unparser
- private lazy val sequenceChildren = sequenceChildrenArg.toVector
+ lazy val sequenceChildren = sequenceChildrenArg.toVector
override lazy val parser: Parser = sq.hasSeparator match {
case true => new OrderedSeparatedSequenceParser(
@@ -148,10 +148,8 @@ class UnorderedSequence(sq: SequenceTermBase, sequenceChildrenArg: Seq[SequenceC
private lazy val sequenceChildren = sequenceChildrenArg.toVector
-
private lazy val parsers = alternatives.map(_.parser)
-
override lazy val parser: Parser = {
lazy val choiceParser = new ChoiceParser(srd, parsers.toVector)
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SpecifiedLength.scala b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SpecifiedLength.scala
index f1934d2..7c112b7 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SpecifiedLength.scala
+++ b/daffodil-core/src/main/scala/org/apache/daffodil/grammar/primitives/SpecifiedLength.scala
@@ -141,7 +141,6 @@ class SpecifiedLengthImplicit(e: ElementBase, eGram: => Gram, nBits: Long)
eParser,
e.elementRuntimeData,
nBits)
-
}
class SpecifiedLengthPrefixed(e: ElementBase, eGram: => Gram, bitsMultiplier: Int)
diff --git a/daffodil-lib/src/main/resources/org/apache/daffodil/xsd/tdml.xsd b/daffodil-lib/src/main/resources/org/apache/daffodil/xsd/tdml.xsd
index 2cd8bfc..8371293 100644
--- a/daffodil-lib/src/main/resources/org/apache/daffodil/xsd/tdml.xsd
+++ b/daffodil-lib/src/main/resources/org/apache/daffodil/xsd/tdml.xsd
@@ -101,6 +101,7 @@
<simpleType name="implementationItem">
<restriction base="xs:token">
<enumeration value="daffodil"/>
+ <enumeration value="daffodil-runtime2"/>
<enumeration value="ibm"/>
</restriction>
</simpleType>
diff --git a/daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/dafext.xsd b/daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/dafext.xsd
index 6a7548c..fa44bf1 100644
--- a/daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/dafext.xsd
+++ b/daffodil-propgen/src/main/resources/org/apache/daffodil/xsd/dafext.xsd
@@ -410,6 +410,14 @@
</xs:documentation>
</xs:annotation>
</xs:element>
+ <xs:element name="tdmlImplementation" type="xs:string" default="daffodil" minOccurs="0">
+ <xs:annotation>
+ <xs:documentation>
+ TDMLDFDLProcessorFactory implementation to use when running TDML tests.
+ Allowed values are "daffodil" (default), "daffodil-runtime2", and "ibm".
+ </xs:documentation>
+ </xs:annotation>
+ </xs:element>
<xs:element name="unqualifiedPathStepPolicy" type="daf:TunableUnqualifiedPathStepPolicy" default="noNamespace" minOccurs="0">
<xs:annotation>
<xs:documentation>
diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/api/DFDLParserUnparser.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/api/DFDLParserUnparser.scala
index b0f27ca..1f0ee1a 100644
--- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/api/DFDLParserUnparser.scala
+++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/api/DFDLParserUnparser.scala
@@ -32,6 +32,7 @@ import org.apache.daffodil.util.Coroutine
import org.apache.daffodil.util.Maybe
import org.apache.daffodil.util.Maybe.Nope
import org.xml.sax.SAXException
+import org.apache.daffodil.xml.RefQName
/**
* This file contains traits that define an abstract API that any DFDL processor
@@ -111,7 +112,7 @@ object DFDL {
def setExternalDFDLVariable(name: String, namespace: String, value: String): Unit
/**
- * Compilation returns a parser factory, which must be interrogated for diagnostics
+ * Compilation returns a [[ProcessorFactory]], which must be interrogated for diagnostics
* to see if compilation was successful or not.
*/
def compileSource(
@@ -123,7 +124,8 @@ object DFDL {
}
/**
- * The point of processor factory is to allow compilation of the path expression.
+ * The point of [[ProcessorFactory]] is to allow compilation of the path expression
+ * and/or generation of source code to process data matching the compiled schema
*/
trait ProcessorFactory extends WithDiagnostics {
@@ -147,11 +149,41 @@ object DFDL {
@deprecated("Use arguments to Compiler.compileSource or compileFile.", "2.6.0")
def setDistinguishedRootNode(name: String, namespace: String = null): Unit
+ /**
+ * Returns a [[DataProcessor]] to process data matching a compiled XPath expression
+ * @param xpath XPath expression in DFDL schema that data should match (you can use only "/" at this time)
+ */
def onPath(xpath: String): DataProcessor
+
+ /**
+ * Returns a [[CodeGenerator]] to generate code from a DFDL schema to parse or unparse data
+ * @param language source language for generated code (you can use only "c" at this time)
+ */
+ def forLanguage(language: String): CodeGenerator
}
- trait DataProcessor extends WithDiagnostics {
+ /**
+ * Source code generation and compilation is performed with a language-specific [[CodeGenerator]],
+ * which must be interrogated for diagnostics to see if each call was successful or not.
+ */
+ trait CodeGenerator extends WithDiagnostics {
+ /**
+ * Generates language-specific code from a DFDL schema to parse or unparse data
+ * @param rootNS one of the top-level elements of the DFDL schema (if not supplied, uses first top-level element)
+ * @param outputDir output directory in which to generate code
+ * @return path of output directory containing generated code
+ */
+ def generateCode(rootNS: Option[RefQName], outputDir: String): os.Path
+
+ /**
+ * Compiles the generated code so it can be used by a TDML test or something else
+ * @param outputDir path of output directory containing generated code
+ * @return path of executable built from generated code
+ */
+ def compileCode(outputDir: os.Path): os.Path
+ }
+ trait DataProcessorBase {
/**
* Returns a data processor with all the same state, but the validation mode changed to that of the argument.
*
@@ -184,7 +216,9 @@ object DFDL {
def setTunable(tunable: String, value: String): Unit
@deprecated("Use withTunables.", "2.6.0")
def setTunables(tunables: Map[String,String]): Unit
+}
+ trait DataProcessor extends DataProcessorBase with WithDiagnostics {
/**
* Creates a new instance of XMLReader for SAX Parsing
*/
diff --git a/daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/DataProcessor.scala b/daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/DataProcessor.scala
index 3d60450..eaadf1b 100644
--- a/daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/DataProcessor.scala
+++ b/daffodil-runtime1/src/main/scala/org/apache/daffodil/processors/DataProcessor.scala
@@ -31,17 +31,15 @@ import java.util.zip.GZIPOutputStream
import scala.collection.immutable.Queue
import scala.collection.mutable
-import org.apache.daffodil.Implicits._
-import org.apache.daffodil.api.ValidationException
-import org.apache.daffodil.api.ValidationFailure
-import org.apache.daffodil.api.ValidationResult
-import org.apache.daffodil.api.Validator
-import org.apache.daffodil.validation.XercesValidatorFactory
-; object INoWarn4 {
+import org.apache.daffodil.Implicits._; object INoWarn4 {
ImplicitsSuppressUnusedImportWarning() }
import org.apache.daffodil.api.DFDL
import org.apache.daffodil.api.DaffodilTunables
+import org.apache.daffodil.api.ValidationException
+import org.apache.daffodil.api.ValidationFailure
import org.apache.daffodil.api.ValidationMode
+import org.apache.daffodil.api.ValidationResult
+import org.apache.daffodil.api.Validator
import org.apache.daffodil.api.WithDiagnostics
import org.apache.daffodil.debugger.Debugger
import org.apache.daffodil.dsom.TunableLimitExceededError
@@ -74,6 +72,7 @@ import org.apache.daffodil.processors.unparsers.UnparseError
import org.apache.daffodil.util.Maybe
import org.apache.daffodil.util.Maybe._
import org.apache.daffodil.util.Misc
+import org.apache.daffodil.validation.XercesValidatorFactory
import org.apache.daffodil.xml.XMLUtils
import org.xml.sax.ContentHandler
import org.xml.sax.DTDHandler
@@ -698,7 +697,6 @@ class DataProcessor private (
UnparseError(Nope, One(state.currentLocation), "Expected no remaining events, but received %s.", ev.get)
}
}
-
}
class ParseResult(dp: DataProcessor, override val resultState: PState)
@@ -709,7 +707,7 @@ class ParseResult(dp: DataProcessor, override val resultState: PState)
* To be successful here, we need to capture parse/validation
* errors and add them to the Diagnostics list in the PState.
*
- * @param bytes the parsed Infoset
+ * @param bytes the XML infoset from the parse
*/
def validateResult(bytes: Array[Byte]): Unit = {
Assert.usage(resultState.processorStatus eq Success)
diff --git a/daffodil-runtime2/src/main/resources/.clang-format b/daffodil-runtime2/src/main/resources/.clang-format
new file mode 100644
index 0000000..57fbb91
--- /dev/null
+++ b/daffodil-runtime2/src/main/resources/.clang-format
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+AlignConsecutiveDeclarations: true
+AllowShortFunctionsOnASingleLine: None
+AlwaysBreakAfterReturnType: TopLevelDefinitions
+BasedOnStyle: llvm
+BreakBeforeBraces: Allman
+IndentWidth: 4
+KeepEmptyLinesAtTheStartOfBlocks: false
diff --git a/daffodil-runtime2/src/main/resources/c/daffodil_argp.c b/daffodil-runtime2/src/main/resources/c/daffodil_argp.c
new file mode 100644
index 0000000..7e27972
--- /dev/null
+++ b/daffodil-runtime2/src/main/resources/c/daffodil_argp.c
@@ -0,0 +1,302 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "daffodil_argp.h" // for daffodil_cli, daffodil_parse_cli, ...
+#include <argp.h> // for argp_state, argp_error, error_t, argp_parse
+#include <stdio.h> // for sprintf
+#include <stdlib.h> // for putenv, NULL
+#include <string.h> // for strlen, strcmp
+
+// Initialize our "daffodil" name and version
+
+const char *argp_program_version = "Apache Daffodil (runtime2) 0.1";
+
+// Initialize our "daffodil parse" CLI options
+
+struct daffodil_parse_cli daffodil_parse = {
+ "xml", // default infoset type
+ "-", // default infile
+ "-", // default outfile
+};
+
+static const struct argp_option parse_options[] = {
+ {"infoset-type", 'I', "<infoset_type>", 0,
+ "Infoset type to output. Must be one of 'xml' or 'null'"},
+
+ {"output", 'o', "<file>", 0,
+ "Write output to a given file. If not given or is -, output is written to "
+ "stdout"},
+
+ {0}};
+
+static error_t parse_handler(int key, char *arg, struct argp_state *state);
+
+static const char parse_args_doc[] = "[infile]";
+
+static const char parse_doc[] =
+ "\n"
+ "Parse a file using a DFDL schema\n"
+ "\n"
+ "Parse Options:"
+ "\v"
+ " Trailing arguments:\n"
+ " infile (not required) input file to parse. "
+ "If not specified, or a value of -, reads from stdin";
+
+static const struct argp parse_argp = {
+ parse_options, // array of CLI options
+ parse_handler, // function to get these CLI options
+ parse_args_doc, // short usage documentation
+ parse_doc, // long help documentation
+};
+
+// Handle callbacks to get our "daffodil parse" CLI options
+
+static error_t
+parse_handler(int key, char *arg, struct argp_state *state)
+{
+ struct daffodil_parse_cli *parse = state->input;
+
+ switch (key)
+ {
+ case 'I':
+ parse->infoset_converter = arg;
+ break;
+
+ case 'o':
+ parse->outfile = arg;
+ break;
+
+ case ARGP_KEY_ARG:
+ if (state->arg_num)
+ {
+ argp_error(state, "too many arguments: %s", arg);
+ }
+ parse->infile = arg;
+ break;
+
+ default:
+ return ARGP_ERR_UNKNOWN;
+ }
+
+ return 0;
+}
+
+// Parse our "daffodil parse" command line interface
+
+static error_t
+parse_daffodil_parse_cli(struct argp_state *state)
+{
+ int argc = state->argc - state->next + 1;
+ char **argv = &state->argv[state->next - 1];
+ char * old_cmd = argv[0];
+ char new_cmd[strlen(state->name) + strlen(" parse") + 1];
+
+ sprintf(new_cmd, "%s parse", state->name);
+ argv[0] = new_cmd;
+
+ error_t status = argp_parse(&parse_argp, argc, argv, ARGP_IN_ORDER, &argc,
+ &daffodil_parse);
+
+ argv[0] = old_cmd;
+ state->next += argc - 1;
+
+ return status;
+}
+
+// Initialize our "daffodil unparse" CLI options
+
+struct daffodil_unparse_cli daffodil_unparse = {
+ "xml", // default infoset type
+ "-", // default infile
+ "-", // default outfile
+};
+
+static const struct argp_option unparse_options[] = {
+ {"infoset-type", 'I', "<infoset_type>", 0,
+ "Infoset type to unparse. Must be 'xml'"},
+
+ {"output", 'o', "<file>", 0,
+ "Write output to file. If not given or is -, output is written to "
+ "standard output"},
+
+ {0}};
+
+static error_t unparse_handler(int key, char *arg, struct argp_state *state);
+
+static const char unparse_args_doc[] = "[infile]";
+
+static const char unparse_doc[] =
+ "\n"
+ "Unparse an infoset file using a DFDL schema\n"
+ "\n"
+ "Unparse Options:"
+ "\v"
+ " Trailing arguments:\n"
+ " infile (not required) input file to unparse. If not specified, or "
+ "a value of -, reads from stdin";
+
+static const struct argp unparse_argp = {
+ unparse_options, // array of CLI options
+ unparse_handler, // function to get these CLI options
+ unparse_args_doc, // short usage documentation
+ unparse_doc, // long help documentation
+};
+
+// Handle callbacks to get our "daffodil unparse" CLI options
+
+static error_t
+unparse_handler(int key, char *arg, struct argp_state *state)
+{
+ struct daffodil_unparse_cli *unparse = state->input;
+
+ switch (key)
+ {
+ case 'I':
+ unparse->infoset_converter = arg;
+ break;
+
+ case 'o':
+ unparse->outfile = arg;
+ break;
+
+ case ARGP_KEY_ARG:
+ if (state->arg_num)
+ {
+ argp_error(state, "too many arguments: %s", arg);
+ }
+ unparse->infile = arg;
+ break;
+
+ default:
+ return ARGP_ERR_UNKNOWN;
+ }
+
+ return 0;
+}
+
+// Parse our "daffodil unparse" command line interface
+
+static error_t
+parse_daffodil_unparse_cli(struct argp_state *state)
+{
+ int argc = state->argc - state->next + 1;
+ char **argv = &state->argv[state->next - 1];
+ char * old_cmd = argv[0];
+ char new_cmd[strlen(state->name) + strlen(" unparse") + 1];
+
+ sprintf(new_cmd, "%s unparse", state->name);
+ argv[0] = new_cmd;
+
+ error_t status = argp_parse(&unparse_argp, argc, argv, ARGP_IN_ORDER, &argc,
+ &daffodil_unparse);
+
+ argv[0] = old_cmd;
+ state->next += argc - 1;
+
+ return status;
+}
+
+// Initialize our "daffodil" CLI options
+
+struct daffodil_cli daffodil_cli = {
+ DAFFODIL_NONE, // default subcommand
+ 0, // default verbosity
+};
+
+static const struct argp_option daffodil_options[] = {
+ {"verbose", 'v', 0, 0, "Increment verbosity level, one level for each -v",
+ -1},
+
+ {0}};
+
+static error_t daffodil_handler(int key, char *arg, struct argp_state *state);
+
+static const char daffodil_args_doc[] = "<subcommand> [SUBCOMMAND_OPTION...]";
+
+static const char daffodil_doc[] =
+ "\n"
+ "Global Options:"
+ "\v"
+ "Subcommands:\n"
+ " parse Parse data to a DFDL infoset\n"
+ " unparse Unparse a DFDL infoset\n"
+ "\n"
+ "Run 'daffodil <subcommand> --help' for subcommand specific options";
+
+static const struct argp daffodil_argp = {
+ daffodil_options, // array of CLI options
+ daffodil_handler, // function to get these CLI options
+ daffodil_args_doc, // short usage documentation
+ daffodil_doc, // long help documentation
+};
+
+// Handle callbacks to get our "daffodil" CLI options
+
+static error_t
+daffodil_handler(int key, char *arg, struct argp_state *state)
+{
+ struct daffodil_cli *cli = state->input;
+ error_t status = 0;
+
+ switch (key)
+ {
+ case 'v':
+ cli->verbosity++;
+ break;
+
+ case ARGP_KEY_ARG:
+ if (strcmp(arg, "parse") == 0)
+ {
+ cli->subcommand = DAFFODIL_PARSE;
+ status = parse_daffodil_parse_cli(state);
+ }
+ else if (strcmp(arg, "unparse") == 0)
+ {
+ cli->subcommand = DAFFODIL_UNPARSE;
+ status = parse_daffodil_unparse_cli(state);
+ }
+ else
+ {
+ argp_error(state, "%s is not a valid subcommand", arg);
+ }
+ break;
+
+ case ARGP_KEY_END:
+ if (cli->subcommand == DAFFODIL_NONE)
+ {
+ argp_error(state, "missing subcommand");
+ }
+ break;
+
+ default:
+ return ARGP_ERR_UNKNOWN;
+ }
+
+ return status;
+}
+
+// Parse our "daffodil" command line interface
+
+error_t
+parse_daffodil_cli(int argc, char **argv)
+{
+ static char *argp_help_fmt = "ARGP_HELP_FMT=no-dup-args-note";
+ putenv(argp_help_fmt); // Do not pass an automatic variable to putenv
+ return argp_parse(&daffodil_argp, argc, argv, ARGP_IN_ORDER, NULL,
+ &daffodil_cli);
+}
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala b/daffodil-runtime2/src/main/resources/c/daffodil_argp.h
similarity index 51%
copy from daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
copy to daffodil-runtime2/src/main/resources/c/daffodil_argp.h
index f0bc051..c870123 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
+++ b/daffodil-runtime2/src/main/resources/c/daffodil_argp.h
@@ -15,20 +15,42 @@
* limitations under the License.
*/
-package org.apache.daffodil.grammar
+#ifndef DAFFODIL_ARGP_H
+#define DAFFODIL_ARGP_H
-import org.apache.daffodil.grammar.primitives.UnicodeByteOrderMark
-import org.apache.daffodil.dsom.Root
+// Parse our "daffodil" command line interface
-trait RootGrammarMixin
- extends LocalElementGrammarMixin // can be repeating if not root
- { self: Root =>
+extern int parse_daffodil_cli(int argc, char **argv);
- final lazy val document = prod("document") {
- schemaDefinitionUnless(isScalar, "The document element cannot be an array.")
- UnicodeByteOrderMark(this) ~ documentElement
- }
+// Get our "daffodil" CLI options
- private def documentElement = enclosedElement
+extern struct daffodil_cli
+{
+ enum daffodil_subcommand
+ {
+ DAFFODIL_NONE,
+ DAFFODIL_PARSE,
+ DAFFODIL_UNPARSE
+ } subcommand;
+ int verbosity;
+} daffodil_cli;
-}
+// Get our "daffodil parse" CLI options
+
+extern struct daffodil_parse_cli
+{
+ const char *infoset_converter;
+ const char *infile;
+ const char *outfile;
+} daffodil_parse;
+
+// Get our "daffodil unparse" CLI options
+
+extern struct daffodil_unparse_cli
+{
+ const char *infoset_converter;
+ const char *infile;
+ const char *outfile;
+} daffodil_unparse;
+
+#endif // DAFFODIL_ARGP_H
diff --git a/daffodil-runtime2/src/main/resources/c/daffodil_main.c b/daffodil-runtime2/src/main/resources/c/daffodil_main.c
new file mode 100644
index 0000000..ce6d62b
--- /dev/null
+++ b/daffodil-runtime2/src/main/resources/c/daffodil_main.c
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "daffodil_argp.h" // for daffodil_cli, parse_daffodil_cli, ...
+#include "infoset.h" // for InfosetBase, walkInfoset, rootElement, ...
+#include "xml_reader.h" // for xmlReaderMethods, XMLReader
+#include "xml_writer.h" // for xmlWriterMethods, XMLWriter
+#include <error.h> // for error
+#include <stdio.h> // for FILE, perror, fclose, fopen, stdin
+#include <stdlib.h> // for exit, EXIT_FAILURE
+#include <string.h> // for strcmp
+
+// Open a file or exit if it can't be opened
+
+static FILE *
+fopen_or_exit(FILE *stream, const char *pathname, const char *mode)
+{
+ if (strcmp(pathname, "-") != 0)
+ {
+ stream = fopen(pathname, mode);
+ if (!stream)
+ {
+ perror("Error opening file: ");
+ exit(EXIT_FAILURE);
+ }
+ }
+ return stream;
+}
+
+// Close a stream or exit if it can't be closed
+
+static void
+fclose_or_exit(FILE *stream, FILE *stdin_stdout)
+{
+ if (stream != stdin_stdout && fclose(stream) != 0)
+ {
+ perror("Error closing file: ");
+ exit(EXIT_FAILURE);
+ }
+}
+
+// Print an error and exit if an error occurred or continue otherwise
+
+static void
+continue_or_exit(const char *error_msg)
+{
+ if (error_msg)
+ {
+ error(EXIT_FAILURE, 0, "%s", error_msg);
+ }
+}
+
+// Define our daffodil program's main entry point
+
+int
+main(int argc, char *argv[])
+{
+ // Parse our "daffodil" command line options
+ int status = parse_daffodil_cli(argc, argv);
+
+ if (status == 0)
+ {
+ FILE * input = stdin;
+ FILE * output = stdout;
+ InfosetBase *root = rootElement();
+
+ if (daffodil_cli.subcommand == DAFFODIL_PARSE)
+ {
+ // Open our input and output files if given as arguments.
+ input = fopen_or_exit(input, daffodil_parse.infile, "r");
+ output = fopen_or_exit(output, daffodil_parse.outfile, "w");
+
+ // Parse the input file into our infoset.
+ PState pstate = {input};
+ const char *error_msg = root->erd->parseSelf(root, &pstate);
+ continue_or_exit(error_msg);
+
+ if (strcmp(daffodil_parse.infoset_converter, "xml") == 0)
+ {
+ // Visit the infoset and print XML from it.
+ XMLWriter xmlWriter = {xmlWriterMethods, output};
+ error_msg = walkInfoset((VisitEventHandler *)&xmlWriter, root);
+ continue_or_exit(error_msg);
+ }
+ else
+ {
+ error(EXIT_FAILURE, 0, "Cannot write infoset type '%s'",
+ daffodil_parse.infoset_converter);
+ }
+ }
+ else if (daffodil_cli.subcommand == DAFFODIL_UNPARSE)
+ {
+ // Open our input and output files if given as arguments.
+ input = fopen_or_exit(input, daffodil_unparse.infile, "r");
+ output = fopen_or_exit(output, daffodil_unparse.outfile, "w");
+
+ if (strcmp(daffodil_unparse.infoset_converter, "xml") == 0)
+ {
+ // Initialize our infoset's values from the XML data.
+ XMLReader xmlReader = {xmlReaderMethods, input, root};
+ const char *error_msg =
+ walkInfoset((VisitEventHandler *)&xmlReader, root);
+ continue_or_exit(error_msg);
+ }
+ else
+ {
+ error(EXIT_FAILURE, 0, "Cannot read infoset type '%s'",
+ daffodil_unparse.infoset_converter);
+ }
+
+ // Unparse our infoset to the output file.
+ UState ustate = {output};
+ const char *error_msg = root->erd->unparseSelf(root, &ustate);
+ continue_or_exit(error_msg);
+ }
+
+ // Close our input and out files if we opened them.
+ fclose_or_exit(input, stdin);
+ fclose_or_exit(output, stdout);
+ }
+
+ return status;
+}
diff --git a/daffodil-runtime2/src/main/resources/c/infoset.c b/daffodil-runtime2/src/main/resources/c/infoset.c
new file mode 100644
index 0000000..f0c8bff
--- /dev/null
+++ b/daffodil-runtime2/src/main/resources/c/infoset.c
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "infoset.h" // for walkInfoset, VisitEventHandler, ...
+#include <string.h> // for memccpy
+
+// get_erd_name, get_erd_xmlns, get_erd_ns - get name and xmlns
+// attribute/value from ERD to use on XML element
+
+const char *
+get_erd_name(const ERD *erd)
+{
+ static char name[9999];
+ char * next = name;
+ char * last = name + sizeof(name) - 1;
+
+ if (next && erd->namedQName.prefix)
+ {
+ next = memccpy(next, erd->namedQName.prefix, 0, last - next);
+ if (next)
+ {
+ --next;
+ }
+ }
+ if (next && erd->namedQName.prefix)
+ {
+ next = memccpy(next, ":", 0, last - next);
+ if (next)
+ {
+ --next;
+ }
+ }
+ if (next)
+ {
+ next = memccpy(next, erd->namedQName.local, 0, last - next);
+ if (next)
+ {
+ --next;
+ }
+ }
+ if (!next)
+ {
+ *last = 0;
+ }
+
+ return name;
+}
+
+const char *
+get_erd_xmlns(const ERD *erd)
+{
+ if (erd->namedQName.ns)
+ {
+ static char xmlns[9999];
+ char * next = xmlns;
+ char * last = xmlns + sizeof(xmlns) - 1;
+
+ next = memccpy(next, "xmlns", 0, last - next);
+ if (next)
+ {
+ --next;
+ }
+
+ if (next && erd->namedQName.prefix)
+ {
+ next = memccpy(next, ":", 0, last - next);
+ if (next)
+ {
+ --next;
+ }
+ }
+ if (next && erd->namedQName.prefix)
+ {
+ next = memccpy(next, erd->namedQName.prefix, 0, last - next);
+ if (next)
+ {
+ --next;
+ }
+ }
+ if (!next)
+ {
+ *last = 0;
+ }
+
+ return xmlns;
+ }
+ else
+ {
+ return NULL;
+ }
+}
+
+const char *
+get_erd_ns(const ERD *erd)
+{
+ return erd->namedQName.ns;
+}
+
+// walkInfosetNode - recursively walk an infoset node and call
+// VisitEventHandler methods
+
+static const char *
+walkInfosetNode(const VisitEventHandler *handler, const InfosetBase *infoNode)
+{
+ const char *error_msg = NULL;
+
+ // Start visiting the node
+ if (!error_msg)
+ {
+ error_msg = handler->visitStartComplex(handler, infoNode);
+ }
+
+ // Walk the node's children recursively
+ const size_t count = infoNode->erd->numChildren;
+ const ERD **const childrenERDs = infoNode->erd->childrenERDs;
+ const ptrdiff_t * offsets = infoNode->erd->offsets;
+
+ for (size_t i = 0; i < count && !error_msg; i++)
+ {
+ ptrdiff_t offset = offsets[i];
+ const ERD *childERD = childrenERDs[i];
+ // We use only one of these variables below depending on typeCode
+ const InfosetBase *childNode =
+ (const InfosetBase *)((const char *)infoNode + offset);
+ const int32_t *intLocation =
+ (const int32_t *)((const char *)infoNode + offset);
+
+ // Need to handle more element types
+ enum TypeCode typeCode = childERD->typeCode;
+ switch (typeCode)
+ {
+ case COMPLEX:
+ error_msg = walkInfosetNode(handler, childNode);
+ break;
+ case PRIMITIVE_INT32:
+ error_msg = handler->visitInt32Elem(handler, childERD, intLocation);
+ break;
+ }
+ }
+
+ // End visiting the node
+ if (!error_msg)
+ {
+ error_msg = handler->visitEndComplex(handler, infoNode);
+ }
+
+ return error_msg;
+}
+
+// walkInfoset - walk an infoset and call VisitEventHandler methods
+
+const char *
+walkInfoset(const VisitEventHandler *handler, const InfosetBase *infoset)
+{
+ const char *error_msg = NULL;
+
+ if (!error_msg)
+ {
+ error_msg = handler->visitStartDocument(handler);
+ }
+ if (!error_msg)
+ {
+ error_msg = walkInfosetNode(handler, infoset);
+ }
+ if (!error_msg)
+ {
+ error_msg = handler->visitEndDocument(handler);
+ }
+
+ return error_msg;
+}
+
+// eof_or_error_msg - check if a stream has its eof or error indicator set
+
+const char *
+eof_or_error_msg(FILE *stream)
+{
+ if (feof(stream))
+ {
+ return "Found eof indicator after reading stream";
+ }
+ else if (ferror(stream))
+ {
+ return "Found error indicator after reading stream";
+ }
+ else
+ {
+ return NULL;
+ }
+}
diff --git a/daffodil-runtime2/src/main/resources/c/infoset.h b/daffodil-runtime2/src/main/resources/c/infoset.h
new file mode 100644
index 0000000..459557c
--- /dev/null
+++ b/daffodil-runtime2/src/main/resources/c/infoset.h
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef INFOSET_H
+#define INFOSET_H
+
+#include <stddef.h> // for ptrdiff_t
+#include <stdint.h> // for int32_t
+#include <stdio.h> // for FILE, size_t
+
+// Prototypes needed for compilation
+
+struct ElementRuntimeData;
+struct InfosetBase;
+struct PState;
+struct UState;
+struct VisitEventHandler;
+
+typedef struct ElementRuntimeData ERD;
+typedef struct InfosetBase InfosetBase;
+typedef struct PState PState;
+typedef struct UState UState;
+typedef struct VisitEventHandler VisitEventHandler;
+
+typedef void (*ERDInitSelf)(InfosetBase *infoNode);
+typedef const char *(*ERDParseSelf)(InfosetBase * infoNode,
+ const PState *pstate);
+typedef const char *(*ERDUnparseSelf)(const InfosetBase *infoNode,
+ const UState * ustate);
+
+typedef const char *(*VisitStartDocument)(const VisitEventHandler *handler);
+typedef const char *(*VisitEndDocument)(const VisitEventHandler *handler);
+typedef const char *(*VisitStartComplex)(const VisitEventHandler *handler,
+ const InfosetBase * base);
+typedef const char *(*VisitEndComplex)(const VisitEventHandler *handler,
+ const InfosetBase * base);
+typedef const char *(*VisitInt32Elem)(const VisitEventHandler *handler,
+ const ERD *erd, const int32_t *location);
+
+// NamedQName - name of an infoset element
+
+typedef struct NamedQName
+{
+ const char *prefix; // prefix (optional, may be NULL)
+ const char *local; // local name
+ const char *ns; // namespace URI (optional, may be NULL)
+} NamedQName;
+
+// TypeCode - type of an infoset element
+
+enum TypeCode
+{
+ COMPLEX,
+ PRIMITIVE_INT32
+};
+
+// ERD - element runtime data needed to parse/unparse objects
+
+typedef struct ElementRuntimeData
+{
+ const NamedQName namedQName;
+ const enum TypeCode typeCode;
+ const size_t numChildren;
+ const ptrdiff_t * offsets;
+ const ERD ** childrenERDs;
+
+ const ERDInitSelf initSelf;
+ const ERDParseSelf parseSelf;
+ const ERDUnparseSelf unparseSelf;
+} ERD;
+
+// InfosetBase - representation of an infoset element
+
+typedef struct InfosetBase
+{
+ const ERD *erd;
+} InfosetBase;
+
+// PState - parser state while parsing input
+
+typedef struct PState
+{
+ FILE *stream; // input to read from
+} PState;
+
+// UState - unparser state while unparsing infoset
+
+typedef struct UState
+{
+ FILE *stream; // output to write to
+} UState;
+
+// VisitEventHandler - methods to be called when walking an infoset
+
+typedef struct VisitEventHandler
+{
+ const VisitStartDocument visitStartDocument;
+ const VisitEndDocument visitEndDocument;
+ const VisitStartComplex visitStartComplex;
+ const VisitEndComplex visitEndComplex;
+ const VisitInt32Elem visitInt32Elem;
+} VisitEventHandler;
+
+// get_erd_name, get_erd_xmlns, get_erd_ns - get name and xmlns
+// attribute/value from ERD to use for XML element
+
+extern const char *get_erd_name(const ERD *erd);
+extern const char *get_erd_xmlns(const ERD *erd);
+extern const char *get_erd_ns(const ERD *erd);
+
+// rootElement - return a root element to walk while parsing or unparsing
+
+// (actual definition will be in generated_code.c, not infoset.c)
+extern InfosetBase *rootElement();
+
+// walkInfoset - walk an infoset and call VisitEventHandler methods
+
+extern const char *walkInfoset(const VisitEventHandler *handler,
+ const InfosetBase * infoset);
+
+// eof_or_error_msg - check if a stream has its eof or error indicator set
+
+extern const char *eof_or_error_msg(FILE *stream);
+
+#endif // INFOSET_H
diff --git a/daffodil-runtime2/src/main/resources/c/stack.c b/daffodil-runtime2/src/main/resources/c/stack.c
new file mode 100644
index 0000000..c1814bb
--- /dev/null
+++ b/daffodil-runtime2/src/main/resources/c/stack.c
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "stack.h"
+#include <error.h> // for error
+#include <stdbool.h> // for bool
+#include <stddef.h> // for ptrdiff_t
+#include <stdlib.h> // for EXIT_FAILURE
+
+// Initialize stack with preallocated array
+
+void
+stack_init(stack_t *p_stack, stack_item_t *p_array, ptrdiff_t capacity)
+{
+ p_stack->p_after = p_array;
+ p_stack->p_array = p_array;
+ p_stack->capacity = capacity;
+}
+
+// Check whether stack is empty
+
+bool
+stack_is_empty(stack_t *p_stack)
+{
+ return p_stack->p_after == p_stack->p_array;
+}
+
+// Check whether stack is full
+
+bool
+stack_is_full(stack_t *p_stack)
+{
+ ptrdiff_t count = p_stack->p_after - p_stack->p_array;
+ return count >= p_stack->capacity;
+}
+
+// Pop element from stack
+
+stack_item_t
+stack_pop(stack_t *p_stack)
+{
+ if (stack_is_empty(p_stack))
+ {
+ error(EXIT_FAILURE, 0, "Stack underflow - program terminated");
+ }
+ return *(--p_stack->p_after);
+}
+
+// Push element into stack
+
+void
+stack_push(stack_t *p_stack, stack_item_t item)
+{
+ if (stack_is_full(p_stack))
+ {
+ error(EXIT_FAILURE, 0, "Stack overflow - program terminated");
+ }
+ *(p_stack->p_after++) = item;
+}
+
+// Get stack's top element
+
+stack_item_t
+stack_top(stack_t *p_stack)
+{
+ if (stack_is_empty(p_stack))
+ {
+ error(EXIT_FAILURE, 0, "Stack empty - program terminated");
+ }
+ return *(p_stack->p_after - 1);
+}
diff --git a/daffodil-runtime2/src/main/resources/c/stack.h b/daffodil-runtime2/src/main/resources/c/stack.h
new file mode 100644
index 0000000..bf1cf3a
--- /dev/null
+++ b/daffodil-runtime2/src/main/resources/c/stack.h
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef STACK_H
+#define STACK_H
+
+#include <mxml.h> // for mxml_node_t
+#include <stdbool.h> // for bool
+#include <stddef.h> // for ptrdiff_t
+
+// Type of element pushed into stack
+
+typedef mxml_node_t *stack_item_t;
+
+// Implement stack using preallocated array
+
+typedef struct
+{
+ stack_item_t *p_after; // Pointer to one past top element
+ stack_item_t *p_array; // Pointer to stack's array
+ ptrdiff_t capacity; // Size of stack's array
+} stack_t;
+
+// Initialize stack with preallocated array
+
+extern void stack_init(stack_t *p_stack, stack_item_t *p_array,
+ ptrdiff_t capacity);
+
+// Check whether stack is empty
+
+extern bool stack_is_empty(stack_t *p_stack);
+
+// Check whether stack is full
+
+extern bool stack_is_full(stack_t *p_stack);
+
+// Pop element from stack
+
+extern stack_item_t stack_pop(stack_t *p_stack);
+
+// Push element into stack
+
+extern void stack_push(stack_t *p_stack, stack_item_t item);
+
+// Get stack's top element
+
+extern stack_item_t stack_top(stack_t *p_stack);
+
+#endif // STACK_H
diff --git a/daffodil-runtime2/src/main/resources/c/xml_reader.c b/daffodil-runtime2/src/main/resources/c/xml_reader.c
new file mode 100644
index 0000000..34035d3
--- /dev/null
+++ b/daffodil-runtime2/src/main/resources/c/xml_reader.c
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "xml_reader.h"
+#include <errno.h> // for errno
+#include <inttypes.h> // for strtoimax
+#include <mxml.h> // for mxmlWalkNext, mxmlGetType, mxmlGetElement, ...
+#include <stdint.h> // for intmax_t, int32_t, INT32_MAX, INT32_MIN
+#include <string.h> // for strcmp, strlen, strncmp
+
+// Convert an XML element's text to an integer (BSD function not
+// widely available, so roll our own function based on strtoimax)
+
+static intmax_t
+strtonum(const char *numptr, intmax_t minval, intmax_t maxval,
+ const char **errstrp)
+{
+ char *endptr = NULL;
+
+ // Clear errno to detect failure after calling strtoimax
+ errno = 0;
+ const intmax_t value = strtoimax(numptr, &endptr, 10);
+
+ // Report any issues converting the string to a number
+ if (errno != 0)
+ {
+ *errstrp = "Error converting XML data to integer";
+ }
+ else if (endptr == numptr)
+ {
+ *errstrp = "Found no number in XML data";
+ }
+ else if (*endptr != '\0')
+ {
+ *errstrp = "Found non-number characters in XML data";
+ }
+ else if (value < minval || value > maxval || maxval < minval)
+ {
+ *errstrp = "Number in XML data out of range";
+ }
+ else
+ {
+ *errstrp = NULL;
+ }
+
+ return value;
+}
+
+// Read XML data from file before walking infoset
+
+static const char *
+xmlStartDocument(XMLReader *reader)
+{
+ // Load the XML data into memory
+ reader->xml = mxmlLoadFile(NULL, reader->stream, MXML_OPAQUE_CALLBACK);
+ reader->node = reader->xml;
+ if (!reader->node)
+ {
+ return "Unable to read XML data from input file";
+ }
+
+ // Consume the <?xml line if there is one
+ const char *name = mxmlGetElement(reader->node);
+ if (name && strncmp(name, "?xml", strlen("?xml")) == 0)
+ {
+ do
+ {
+ reader->node =
+ mxmlWalkNext(reader->node, reader->xml, MXML_DESCEND);
+ } while (mxmlGetType(reader->node) == MXML_OPAQUE);
+ name = mxmlGetElement(reader->node);
+ }
+
+ // Consume a comment if there is one
+ if (name && strncmp(name, "!--", strlen("!--")) == 0)
+ {
+ do
+ {
+ reader->node =
+ mxmlWalkNext(reader->node, reader->xml, MXML_DESCEND);
+ } while (mxmlGetType(reader->node) == MXML_OPAQUE);
+ }
+
+ return reader->node ? NULL : "Ran out of XML data";
+}
+
+// Delete XML data after walking infoset
+
+static const char *
+xmlEndDocument(XMLReader *reader)
+{
+ // Consume any remaining newlines or whitespace
+ while (mxmlGetType(reader->node) == MXML_OPAQUE)
+ {
+ reader->node = mxmlWalkNext(reader->node, reader->xml, MXML_DESCEND);
+ }
+
+ // Check whether we have consumed all of the XML data
+ if (reader->node)
+ {
+ // This code path exits the program - no need to call mxmlDelete
+ return "Did not consume all of the XML data";
+ }
+
+ // Free the storage allocated to hold the XML data
+ mxmlDelete(reader->xml);
+ reader->xml = NULL;
+ reader->node = NULL;
+ return NULL;
+}
+
+// Continue walking both XML data and infoset in lockstep
+
+static const char *
+xmlStartComplex(XMLReader *reader, const InfosetBase *base)
+{
+ // Consume any newlines or whitespace before the element
+ while (mxmlGetType(reader->node) == MXML_OPAQUE)
+ {
+ reader->node = mxmlWalkNext(reader->node, reader->xml, MXML_DESCEND);
+ }
+
+ // Get the element and consume it
+ const char *name_from_xml = mxmlGetElement(reader->node);
+ const char *name_from_erd = get_erd_name(base->erd);
+ reader->node = mxmlWalkNext(reader->node, reader->xml, MXML_DESCEND);
+
+ // Check whether we are walking both XML data and infoset in lockstep
+ if (name_from_xml && name_from_erd)
+ {
+ return strcmp(name_from_xml, name_from_erd) == 0
+ ? NULL
+ : "Found mismatch between XML data and infoset";
+ }
+ else
+ {
+ return "Ran out of XML data";
+ }
+}
+
+// Consume XML data only on start events, not end events
+
+static const char *
+xmlEndComplex(XMLReader *reader, const InfosetBase *base)
+{
+ (void)reader;
+ (void)base;
+ return NULL;
+}
+
+// Read 32-bit integer value from XML data
+
+static const char *
+xmlInt32Elem(XMLReader *reader, const ERD *erd, int32_t *location)
+{
+ // Consume any newlines or whitespace before the element
+ while (mxmlGetType(reader->node) == MXML_OPAQUE)
+ {
+ reader->node = mxmlWalkNext(reader->node, reader->xml, MXML_DESCEND);
+ }
+
+ // Get the element and consume it
+ const char *name_from_xml = mxmlGetElement(reader->node);
+ const char *name_from_erd = get_erd_name(erd);
+ const char *number_from_xml = mxmlGetOpaque(reader->node);
+ reader->node = mxmlWalkNext(reader->node, reader->xml, MXML_DESCEND);
+
+ // Check whether we are walking both XML data and infoset in lockstep
+ if (name_from_xml && name_from_erd)
+ {
+ if (strcmp(name_from_xml, name_from_erd) == 0)
+ {
+ // Check for any errors getting the 32-bit integer
+ const char *errstr = NULL;
+ *location = (int32_t)strtonum(number_from_xml, INT32_MIN, INT32_MAX,
+ &errstr);
+ return errstr;
+ }
+ else
+ {
+ return "Found mismatch between XML data and infoset";
+ }
+ }
+ else
+ {
+ return "Ran out of XML data";
+ }
+}
+
+// Initialize a struct with our visitor event handler methods
+
+const VisitEventHandler xmlReaderMethods = {
+ (VisitStartDocument)&xmlStartDocument, (VisitEndDocument)&xmlEndDocument,
+ (VisitStartComplex)&xmlStartComplex, (VisitEndComplex)&xmlEndComplex,
+ (VisitInt32Elem)&xmlInt32Elem,
+};
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala b/daffodil-runtime2/src/main/resources/c/xml_reader.h
similarity index 58%
copy from daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
copy to daffodil-runtime2/src/main/resources/c/xml_reader.h
index f0bc051..3333f55 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
+++ b/daffodil-runtime2/src/main/resources/c/xml_reader.h
@@ -15,20 +15,26 @@
* limitations under the License.
*/
-package org.apache.daffodil.grammar
+#ifndef XML_READER_H
+#define XML_READER_H
-import org.apache.daffodil.grammar.primitives.UnicodeByteOrderMark
-import org.apache.daffodil.dsom.Root
+#include "infoset.h" // for VisitEventHandler, InfosetBase
+#include <mxml.h> // for mxml_node_t
+#include <stdio.h> // for FILE
-trait RootGrammarMixin
- extends LocalElementGrammarMixin // can be repeating if not root
- { self: Root =>
+// XMLReader - infoset visitor with methods to read XML
- final lazy val document = prod("document") {
- schemaDefinitionUnless(isScalar, "The document element cannot be an array.")
- UnicodeByteOrderMark(this) ~ documentElement
- }
+typedef struct XMLReader
+{
+ const VisitEventHandler handler;
+ FILE * stream;
+ InfosetBase * root;
+ mxml_node_t * xml;
+ mxml_node_t * node;
+} XMLReader;
- private def documentElement = enclosedElement
+// XMLReader methods to pass to walkInfoset method
-}
+extern const VisitEventHandler xmlReaderMethods;
+
+#endif // XML_READER_H
diff --git a/daffodil-runtime2/src/main/resources/c/xml_writer.c b/daffodil-runtime2/src/main/resources/c/xml_writer.c
new file mode 100644
index 0000000..2db0ce4
--- /dev/null
+++ b/daffodil-runtime2/src/main/resources/c/xml_writer.c
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "xml_writer.h"
+#include "stack.h" // for stack_is_empty, stack_pop, stack_push, stack_top
+#include <assert.h> // for assert
+#include <mxml.h> // for mxml_node_t, mxmlNewElement, mxmlNewOpaquef, ...
+#include <stdint.h> // for int32_t
+#include <stdio.h> // for NULL, fflush
+
+// Push new XML document on stack. This function is not
+// thread-safe since it uses static storage.
+
+static const char *
+xmlStartDocument(XMLWriter *writer)
+{
+#define MAX_DEPTH 100
+ static mxml_node_t *array[MAX_DEPTH];
+ stack_init(&writer->stack, array, MAX_DEPTH);
+
+ mxml_node_t *xml = mxmlNewXML("1.0");
+ stack_push(&writer->stack, xml);
+ return xml ? NULL : "Error making new XML declaration";
+}
+
+// Pop completed XML document off stack and write it to stream
+
+static const char *
+xmlEndDocument(XMLWriter *writer)
+{
+ mxml_node_t *xml = stack_pop(&writer->stack);
+ assert(stack_is_empty(&writer->stack));
+ int status = mxmlSaveFile(xml, writer->stream, MXML_NO_CALLBACK);
+ if (status < 0)
+ {
+ return "Error writing XML document";
+ }
+ status = fflush(writer->stream);
+ mxmlDelete(xml);
+ return status == 0 ? NULL : "Error flushing stream";
+}
+
+// Push new complex element on stack
+
+static const char *
+xmlStartComplex(XMLWriter *writer, const InfosetBase *base)
+{
+ mxml_node_t *complex = NULL;
+ if (!stack_is_full(&writer->stack))
+ {
+ mxml_node_t *parent = stack_top(&writer->stack);
+ const char * name = get_erd_name(base->erd);
+ const char * xmlns = get_erd_xmlns(base->erd);
+ complex = mxmlNewElement(parent, name);
+ if (xmlns)
+ {
+ const char *ns = get_erd_ns(base->erd);
+ mxmlElementSetAttr(complex, xmlns, ns);
+ }
+ stack_push(&writer->stack, complex);
+ }
+ return complex ? NULL : "Error making new complex element";
+}
+
+// Pop completed complex element off stack
+
+static const char *
+xmlEndComplex(XMLWriter *writer, const InfosetBase *base)
+{
+ mxml_node_t *complex = NULL;
+ if (!stack_is_empty(&writer->stack))
+ {
+ complex = stack_pop(&writer->stack);
+ (void)base;
+ }
+ return complex ? NULL : "Underflowed the XML stack";
+}
+
+// Write 32-bit integer value as element
+
+static const char *
+xmlInt32Elem(XMLWriter *writer, const ERD *erd, const int32_t *location)
+{
+ mxml_node_t *parent = stack_top(&writer->stack);
+ const char * name = get_erd_name(erd);
+ const char * xmlns = get_erd_xmlns(erd);
+ mxml_node_t *simple = mxmlNewElement(parent, name);
+ int32_t value = *location;
+ mxml_node_t *text = mxmlNewOpaquef(simple, "%i", value);
+ if (xmlns)
+ {
+ const char *ns = get_erd_ns(erd);
+ mxmlElementSetAttr(simple, xmlns, ns);
+ }
+ return (simple && text) ? NULL : "Error making new int32 simple element";
+}
+
+// Initialize a struct with our visitor event handler methods
+
+const VisitEventHandler xmlWriterMethods = {
+ (VisitStartDocument)&xmlStartDocument, (VisitEndDocument)&xmlEndDocument,
+ (VisitStartComplex)&xmlStartComplex, (VisitEndComplex)&xmlEndComplex,
+ (VisitInt32Elem)&xmlInt32Elem,
+};
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala b/daffodil-runtime2/src/main/resources/c/xml_writer.h
similarity index 62%
copy from daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
copy to daffodil-runtime2/src/main/resources/c/xml_writer.h
index f0bc051..45d7a45 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
+++ b/daffodil-runtime2/src/main/resources/c/xml_writer.h
@@ -15,20 +15,24 @@
* limitations under the License.
*/
-package org.apache.daffodil.grammar
+#ifndef XML_WRITER_H
+#define XML_WRITER_H
-import org.apache.daffodil.grammar.primitives.UnicodeByteOrderMark
-import org.apache.daffodil.dsom.Root
+#include "infoset.h" // for VisitEventHandler
+#include "stack.h" // for stack_t
+#include <stdio.h> // for FILE
-trait RootGrammarMixin
- extends LocalElementGrammarMixin // can be repeating if not root
- { self: Root =>
+// XMLWriter - infoset visitor with methods to output XML
- final lazy val document = prod("document") {
- schemaDefinitionUnless(isScalar, "The document element cannot be an array.")
- UnicodeByteOrderMark(this) ~ documentElement
- }
+typedef struct XMLWriter
+{
+ const VisitEventHandler handler;
+ FILE * stream;
+ stack_t stack;
+} XMLWriter;
- private def documentElement = enclosedElement
+// XMLWriter methods to pass to walkInfoset method
-}
+extern const VisitEventHandler xmlWriterMethods;
+
+#endif // XML_WRITER_H
diff --git a/daffodil-runtime2/src/main/resources/examples/ex_int32.c b/daffodil-runtime2/src/main/resources/examples/ex_int32.c
new file mode 100644
index 0000000..d2b9217
--- /dev/null
+++ b/daffodil-runtime2/src/main/resources/examples/ex_int32.c
@@ -0,0 +1,260 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "ex_int32.h" // for generated code structs
+#include <endian.h> // for be32toh, htobe32
+#include <stddef.h> // for ptrdiff_t
+#include <stdio.h> // for NULL, fread, fwrite, size_t, FILE
+
+// Prototypes needed for compilation
+
+static void c2_initSelf(c2 *instance);
+static const char *c2_parseSelf(c2 *instance, const PState *pstate);
+static const char *c2_unparseSelf(const c2 *instance, const UState *ustate);
+static void c1_initSelf(c1 *instance);
+static const char *c1_parseSelf(c1 *instance, const PState *pstate);
+static const char *c1_unparseSelf(const c1 *instance, const UState *ustate);
+
+// Metadata singletons
+
+static const ERD e1_ERD = {
+ {
+ "ex", // namedQName.prefix
+ "e1", // namedQName.local
+ NULL, // namedQName.ns
+ },
+ PRIMITIVE_INT32, // typeCode
+ 0, // numChildren
+ NULL, // offsets
+ NULL, // childrenERDs
+ NULL, // initSelf
+ NULL, // parseSelf
+ NULL, // unparseSelf
+};
+
+static const ERD e2_ERD = {
+ {
+ "ex", // namedQName.prefix
+ "e2", // namedQName.local
+ NULL, // namedQName.ns
+ },
+ PRIMITIVE_INT32, // typeCode
+ 0, // numChildren
+ NULL, // offsets
+ NULL, // childrenERDs
+ NULL, // initSelf
+ NULL, // parseSelf
+ NULL, // unparseSelf
+};
+
+static const ERD e3_ERD = {
+ {
+ "ex", // namedQName.prefix
+ "e3", // namedQName.local
+ NULL, // namedQName.ns
+ },
+ PRIMITIVE_INT32, // typeCode
+ 0, // numChildren
+ NULL, // offsets
+ NULL, // childrenERDs
+ NULL, // initSelf
+ NULL, // parseSelf
+ NULL, // unparseSelf
+};
+
+static const c2 c2_compute_ERD_offsets;
+
+static const ptrdiff_t c2_offsets[2] = {
+ (char *)&c2_compute_ERD_offsets.e2 - (char *)&c2_compute_ERD_offsets,
+ (char *)&c2_compute_ERD_offsets.e3 - (char *)&c2_compute_ERD_offsets};
+
+static const ERD *c2_childrenERDs[2] = {&e2_ERD, &e3_ERD};
+
+static const ERD c2_ERD = {
+ {
+ "ex", // namedQName.prefix
+ "c2", // namedQName.local
+ NULL, // namedQName.ns
+ },
+ COMPLEX, // typeCode
+ 2, // numChildren
+ c2_offsets, // offsets
+ c2_childrenERDs, // childrenERDs
+ (ERDInitSelf)&c2_initSelf, // initSelf
+ (ERDParseSelf)&c2_parseSelf, // parseSelf
+ (ERDUnparseSelf)&c2_unparseSelf, // unparseSelf
+};
+
+static const c1 c1_compute_ERD_offsets;
+
+static const ptrdiff_t c1_offsets[2] = {
+ (char *)&c1_compute_ERD_offsets.e1 - (char *)&c1_compute_ERD_offsets,
+ (char *)&c1_compute_ERD_offsets.c2 - (char *)&c1_compute_ERD_offsets};
+
+static const ERD *c1_childrenERDs[2] = {&e1_ERD, &c2_ERD};
+
+static const ERD c1_ERD = {
+ {
+ "ex", // namedQName.prefix
+ "c1", // namedQName.local
+ "http://example.com", // namedQName.ns
+ },
+ COMPLEX, // typeCode
+ 2, // numChildren
+ c1_offsets, // offsets
+ c1_childrenERDs, // childrenERDs
+ (ERDInitSelf)&c1_initSelf, // initSelf
+ (ERDParseSelf)&c1_parseSelf, // parseSelf
+ (ERDUnparseSelf)&c1_unparseSelf, // unparseSelf
+};
+
+// Return a root element to be used for parsing or unparsing
+
+InfosetBase *
+rootElement()
+{
+ static c1 instance;
+ InfosetBase *root = &instance._base;
+ c1_ERD.initSelf(root);
+ return root;
+}
+
+// Methods to initialize, parse, and unparse infoset nodes
+
+static void
+c2_initSelf(c2 *instance)
+{
+ instance->e2 = 0xCDCDCDCD;
+ instance->e3 = 0xCDCDCDCD;
+ instance->_base.erd = &c2_ERD;
+}
+
+static const char *
+c2_parseSelf(c2 *instance, const PState *pstate)
+{
+ const char *error_msg = NULL;
+ if (!error_msg)
+ {
+ char buffer[4];
+ size_t count = fread(&buffer, 1, sizeof(buffer), pstate->stream);
+ if (count < sizeof(buffer))
+ {
+ error_msg = eof_or_error_msg(pstate->stream);
+ }
+ instance->e2 = be32toh(*((uint32_t *)(&buffer)));
+ }
+ if (!error_msg)
+ {
+ char buffer[4];
+ size_t count = fread(&buffer, 1, sizeof(buffer), pstate->stream);
+ if (count < sizeof(buffer))
+ {
+ error_msg = eof_or_error_msg(pstate->stream);
+ }
+ instance->e3 = be32toh(*((uint32_t *)(&buffer)));
+ }
+ return error_msg;
+}
+
+static const char *
+c2_unparseSelf(const c2 *instance, const UState *ustate)
+{
+ const char *error_msg = NULL;
+ if (!error_msg)
+ {
+ union
+ {
+ char c_val[4];
+ uint32_t i_val;
+ } buffer;
+ buffer.i_val = htobe32(instance->e2);
+ size_t count = fwrite(buffer.c_val, 1, sizeof(buffer), ustate->stream);
+ if (count < sizeof(buffer))
+ {
+ error_msg = eof_or_error_msg(ustate->stream);
+ }
+ }
+ if (!error_msg)
+ {
+ union
+ {
+ char c_val[4];
+ uint32_t i_val;
+ } buffer;
+ buffer.i_val = htobe32(instance->e3);
+ size_t count = fwrite(buffer.c_val, 1, sizeof(buffer), ustate->stream);
+ if (count < sizeof(buffer))
+ {
+ error_msg = eof_or_error_msg(ustate->stream);
+ }
+ }
+ return error_msg;
+}
+
+static void
+c1_initSelf(c1 *instance)
+{
+ instance->e1 = 0xCDCDCDCD;
+ c2_initSelf(&instance->c2);
+ instance->_base.erd = &c1_ERD;
+}
+
+static const char *
+c1_parseSelf(c1 *instance, const PState *pstate)
+{
+ const char *error_msg = NULL;
+ if (!error_msg)
+ {
+ char buffer[4];
+ size_t count = fread(&buffer, 1, sizeof(buffer), pstate->stream);
+ if (count < sizeof(buffer))
+ {
+ error_msg = eof_or_error_msg(pstate->stream);
+ }
+ instance->e1 = be32toh(*((uint32_t *)(&buffer)));
+ }
+ if (!error_msg)
+ {
+ error_msg = c2_parseSelf(&instance->c2, pstate);
+ }
+ return error_msg;
+}
+
+static const char *
+c1_unparseSelf(const c1 *instance, const UState *ustate)
+{
+ const char *error_msg = NULL;
+ if (!error_msg)
+ {
+ union
+ {
+ char c_val[4];
+ uint32_t i_val;
+ } buffer;
+ buffer.i_val = htobe32(instance->e1);
+ size_t count = fwrite(buffer.c_val, 1, sizeof(buffer), ustate->stream);
+ if (count < sizeof(buffer))
+ {
+ error_msg = eof_or_error_msg(ustate->stream);
+ }
+ }
+ if (!error_msg)
+ {
+ error_msg = c2_unparseSelf(&instance->c2, ustate);
+ }
+ return error_msg;
+}
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala b/daffodil-runtime2/src/main/resources/examples/ex_int32.h
similarity index 62%
copy from daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
copy to daffodil-runtime2/src/main/resources/examples/ex_int32.h
index f0bc051..affd8b4 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
+++ b/daffodil-runtime2/src/main/resources/examples/ex_int32.h
@@ -15,20 +15,26 @@
* limitations under the License.
*/
-package org.apache.daffodil.grammar
+#ifndef GENERATED_CODE_H
+#define GENERATED_CODE_H
-import org.apache.daffodil.grammar.primitives.UnicodeByteOrderMark
-import org.apache.daffodil.dsom.Root
+#include "infoset.h" // for InfosetBase
+#include <stdint.h> // for int32_t
-trait RootGrammarMixin
- extends LocalElementGrammarMixin // can be repeating if not root
- { self: Root =>
+// Define some infoset structures
- final lazy val document = prod("document") {
- schemaDefinitionUnless(isScalar, "The document element cannot be an array.")
- UnicodeByteOrderMark(this) ~ documentElement
- }
+typedef struct c2
+{
+ InfosetBase _base;
+ int32_t e2;
+ int32_t e3;
+} c2;
- private def documentElement = enclosedElement
+typedef struct c1
+{
+ InfosetBase _base;
+ int32_t e1;
+ c2 c2;
+} c1;
-}
+#endif // GENERATED_CODE_H
diff --git a/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/CodeGenerator.scala b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/CodeGenerator.scala
new file mode 100644
index 0000000..6dcc19d
--- /dev/null
+++ b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/CodeGenerator.scala
@@ -0,0 +1,179 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.runtime2
+
+import java.io.File
+import java.nio.file.FileSystems
+import java.nio.file.Files
+import java.nio.file.Paths
+import java.util.Collections
+
+import org.apache.daffodil.api.DFDL
+import org.apache.daffodil.api.Diagnostic
+import org.apache.daffodil.dsom.Root
+import org.apache.daffodil.dsom.SchemaDefinitionError
+import org.apache.daffodil.runtime2.generators.CodeGeneratorState
+import org.apache.daffodil.util.Misc
+import org.apache.daffodil.xml.RefQName
+
+/**
+ * Generates and compiles C source files from a DFDL schema encapsulated in a [[Root]].
+ * Implements the [[DFDL.CodeGenerator]] trait to allow it to be called by Daffodil code.
+ * @param root Provides the DFDL schema for code generation
+ */
+class CodeGenerator(root: Root) extends DFDL.CodeGenerator {
+ // Used by compileCode and pickCompiler methods
+ private lazy val isWindows = System.getProperty("os.name").toLowerCase().startsWith("windows")
+ // Used by WithDiagnostics methods
+ private var diagnostics: Seq[Diagnostic] = Nil
+ private var errorStatus: Boolean = false
+
+ // Writes C source files into a "c" subdirectory of the given output directory.
+ // Removes the "c" subdirectory if it existed before.
+ override def generateCode(rootNS: Option[RefQName], outputDirArg: String): os.Path = {
+ // Get the paths of the output directory and its code subdirectory
+ val outputDir = os.Path(Paths.get(outputDirArg).toAbsolutePath)
+ val codeDir = outputDir/"c"
+
+ // Ensure our output directory exists while our code subdirectory does not
+ os.makeDir.all(outputDir)
+ os.remove.all(codeDir)
+
+ // Copy our resource directory and all its C source files to our code subdirectory
+ val resourceUri = Misc.getRequiredResource("/c")
+ val fileSystem = if (resourceUri.getScheme == "jar") {
+ val env: java.util.Map[String, String] = Collections.emptyMap()
+ FileSystems.newFileSystem(resourceUri, env)
+ } else {
+ null
+ }
+ try {
+ val resourceDir = os.Path(if (fileSystem != null) fileSystem.getPath("/c") else Paths.get(resourceUri))
+ os.copy(resourceDir, codeDir)
+ }
+ finally
+ if (fileSystem != null) fileSystem.close()
+
+ // Generate C code from the DFDL schema
+ val rootElementName = rootNS.getOrElse(root.refQName).local
+ val codeGeneratorState = new CodeGeneratorState()
+ Runtime2CodeGenerator.generateCode(root.document, codeGeneratorState)
+ val codeHeaderText = codeGeneratorState.generateCodeHeader
+ val codeFileText = codeGeneratorState.generateCodeFile(rootElementName)
+
+ // Write the generated C code into our code subdirectory
+ val generatedCodeHeader = codeDir/"generated_code.h"
+ val generatedCodeFile = codeDir/"generated_code.c"
+ os.write(generatedCodeHeader, codeHeaderText)
+ os.write(generatedCodeFile, codeFileText)
+
+ // Return our output directory in case caller wants to call compileCode next
+ outputDir
+ }
+
+ // Compiles any C source files inside a "c" subdirectory of the given output directory.
+ // Returns the path of the newly created executable to use in TDML tests or something else.
+ override def compileCode(outputDir: os.Path): os.Path = {
+ // Get the paths of the code subdirectory and the executable we will build
+ val codeDir = outputDir/"c"
+ val exe = if (isWindows) codeDir/"daffodil.exe" else codeDir/"daffodil"
+
+ try {
+ // Assemble the compiler's command line arguments
+ val compiler = pickCompiler
+ val files = os.list(codeDir).filter(_.ext == "c")
+ val libs = Seq("-lmxml", if (isWindows) "-largp" else "-lpthread")
+
+ // Call the compiler if it was found. We run the compiler in the output directory,
+ // not in the "c" subdirectory, in order to let the compiler (which might be "zig cc")
+ // cache/reuse previously built files (which might be in a "zig_cache" subdirectory).
+ // We can't let "zig_cache" be put into "c" because we always remove and re-generate
+ // everything in "c" from scratch.
+ if (compiler.nonEmpty) {
+ val result = os.proc(compiler, "-I", codeDir, files, libs, "-o", exe).call(cwd = outputDir, stderr = os.Pipe)
+
+ // Report any compiler output as a warning
+ if (result.out.text.nonEmpty || result.err.text.nonEmpty) {
+ warning("Unexpected compiler output on stdout: %s on stderr: %s", result.out.text, result.err.text)
+ }
+ }
+ } catch {
+ // Report any subprocess termination error as an error
+ case e: os.SubprocessException =>
+ error("Error compiling generated code: %s wd: %s", Misc.getSomeMessage(e).get, outputDir.toString)
+ }
+
+ // Report any failure to build the executable as an error
+ if (!os.exists(exe)) error("No executable was built: %s", exe.toString)
+ exe
+ }
+
+ /**
+ * Searches for any available C compiler on the system. Tries to find the
+ * compiler given by `CC` if `CC` exists in the environment, then tries to
+ * find any compiler from the following list:
+ *
+ * - zig cc
+ * - gcc
+ * - clang
+ * - cc
+ *
+ * Returns the first compiler found as a sequence of strings in case the
+ * compiler is a program with a subcommand argument. Returns the empty
+ * sequence if no compiler could be found in the user's PATH.
+ */
+ lazy val pickCompiler: Seq[String] = {
+ val ccEnv = System.getenv("CC")
+ val compilers = Seq(ccEnv, "zig cc", "gcc", "clang", "cc")
+ val path = System.getenv("PATH").split(File.pathSeparatorChar)
+ def inPath(compiler: String): Boolean = {
+ (compiler != null) && {
+ val exec = compiler.takeWhile(_ != ' ')
+ val exec2 = exec + ".exe"
+ path.exists(dir => Files.isExecutable(Paths.get(dir, exec))
+ || (isWindows && Files.isExecutable(Paths.get(dir, exec2))))
+ }
+ }
+ val compiler = compilers.find(inPath)
+ if (compiler.isDefined)
+ compiler.get.split(' ').toSeq
+ else
+ Seq.empty[String]
+ }
+
+ /**
+ * Adds a warning message to the diagnostics
+ */
+ def warning(formatString: String, args: Any*): Unit = {
+ val sde = new SchemaDefinitionError(None, None, formatString, args: _*)
+ diagnostics :+= sde
+ }
+
+ /**
+ * Adds an error message to the diagnostics and sets isError true
+ */
+ def error(formatString: String, args: Any*): Unit = {
+ val sde = new SchemaDefinitionError(None, None, formatString, args: _*)
+ diagnostics :+= sde
+ errorStatus = true
+ }
+
+ // Implements the WithDiagnostics methods
+ override def getDiagnostics: Seq[Diagnostic] = diagnostics
+ override def isError: Boolean = errorStatus
+}
diff --git a/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/Runtime2CodeGenerator.scala b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/Runtime2CodeGenerator.scala
new file mode 100644
index 0000000..fbf704f
--- /dev/null
+++ b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/Runtime2CodeGenerator.scala
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.runtime2
+
+import org.apache.daffodil.grammar.Gram
+import org.apache.daffodil.grammar.Prod
+import org.apache.daffodil.grammar.RootGrammarMixin
+import org.apache.daffodil.grammar.SeqComp
+import org.apache.daffodil.grammar.primitives.BinaryIntegerKnownLength
+import org.apache.daffodil.grammar.primitives.CaptureContentLengthEnd
+import org.apache.daffodil.grammar.primitives.CaptureContentLengthStart
+import org.apache.daffodil.grammar.primitives.CaptureValueLengthEnd
+import org.apache.daffodil.grammar.primitives.CaptureValueLengthStart
+import org.apache.daffodil.grammar.primitives.ElementCombinator
+import org.apache.daffodil.grammar.primitives.ElementParseAndUnspecifiedLength
+import org.apache.daffodil.grammar.primitives.OrderedSequence
+import org.apache.daffodil.grammar.primitives.ScalarOrderedSequenceChild
+import org.apache.daffodil.grammar.primitives.SpecifiedLengthImplicit
+import org.apache.daffodil.runtime2.generators.BinaryIntegerKnownLengthCodeGenerator
+import org.apache.daffodil.runtime2.generators.CodeGeneratorState
+import org.apache.daffodil.runtime2.generators.ElementParseAndUnspecifiedLengthCodeGenerator
+import org.apache.daffodil.runtime2.generators.OrderedSequenceCodeGenerator
+import org.apache.daffodil.runtime2.generators.SeqCompCodeGenerator
+import org.apache.daffodil.util.Misc
+
+import scala.annotation.tailrec
+
+object Runtime2CodeGenerator
+ extends BinaryIntegerKnownLengthCodeGenerator
+ with ElementParseAndUnspecifiedLengthCodeGenerator
+ with OrderedSequenceCodeGenerator
+ with SeqCompCodeGenerator {
+
+ @tailrec
+ def generateCode(gram: Gram, state: CodeGeneratorState): Unit =
+ gram match {
+ case g: RootGrammarMixin => Runtime2CodeGenerator.generateCode(g.documentElement, state)
+ case g: Prod if (g.guard) => Runtime2CodeGenerator.generateCode(g.gram, state)
+ case g: ElementCombinator => Runtime2CodeGenerator.generateCode(g.subComb, state)
+ case g: SpecifiedLengthImplicit => Runtime2CodeGenerator.generateCode(g.eGram, state)
+ case g: ScalarOrderedSequenceChild => Runtime2CodeGenerator.generateCode(g.term.termContentBody, state)
+ case g: SeqComp => seqCompGenerateCode(g, state)
+ case g: OrderedSequence => orderedSequenceGenerateCode(g, state)
+ case g: ElementParseAndUnspecifiedLength => elementParseAndUnspecifiedLengthGenerateCode(g, state)
+ case g: BinaryIntegerKnownLength => binaryIntegerKnownLengthGenerateCode(g, state)
+ case _: CaptureContentLengthStart => // not generating code here
+ case _: CaptureContentLengthEnd => // not generating code here
+ case _: CaptureValueLengthStart => //not generating code here
+ case _: CaptureValueLengthEnd => // not generating code here
+ case _ => gram.SDE("Code generation not supported for: %s", Misc.getNameFromClass(gram))
+ }
+}
diff --git a/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/Runtime2DataProcessor.scala b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/Runtime2DataProcessor.scala
new file mode 100644
index 0000000..7dd8f68
--- /dev/null
+++ b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/Runtime2DataProcessor.scala
@@ -0,0 +1,206 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.runtime2
+
+import java.io.File
+import java.io.InputStream
+import java.io.OutputStream
+
+import org.apache.daffodil.api.DFDL
+import org.apache.daffodil.api.DaffodilTunables
+import org.apache.daffodil.api.DataLocation
+import org.apache.daffodil.api.ValidationMode
+import org.apache.daffodil.externalvars.Binding
+import org.apache.daffodil.processors.Failure
+import org.apache.daffodil.processors.ProcessorResult
+import org.apache.daffodil.processors.Success
+import org.apache.daffodil.processors.VariableMap
+import org.apache.daffodil.processors.WithDiagnosticsImpl
+import org.apache.daffodil.processors.parsers.ParseError
+import org.apache.daffodil.processors.unparsers.UnparseError
+import org.apache.daffodil.util.Maybe
+import org.apache.daffodil.util.Maybe.Nope
+
+/**
+ * Effectively a scala proxy object that does its work via the underlying C-code.
+ * Will need to consider how to use features of underlying C-code to get infoset,
+ * walk infoset, generate XML for use by TDML tests.
+ */
+class Runtime2DataProcessor(executableFile: os.Path) extends DFDL.DataProcessorBase {
+
+ override def withValidationMode(mode: ValidationMode.Type): DFDL.DataProcessor = ???
+
+ override def withTunable(name: String, value: String): DFDL.DataProcessor = ???
+
+ override def withTunables(tunables: Map[String, String]): DFDL.DataProcessor = ???
+
+ override def withExternalVariables(extVars: Map[String, String]): DFDL.DataProcessor = ???
+
+ override def withExternalVariables(extVars: File): DFDL.DataProcessor = ???
+
+ override def withExternalVariables(extVars: Seq[Binding]): DFDL.DataProcessor = ???
+
+ override def validationMode: ValidationMode.Type = ???
+
+ override def getTunables(): DaffodilTunables = ???
+
+ override def save(output: DFDL.Output): Unit = ???
+
+ override def variableMap: VariableMap = ???
+
+ override def setValidationMode(mode: ValidationMode.Type): Unit = ???
+
+ override def setExternalVariables(extVars: Map[String, String]): Unit = ???
+
+ override def setExternalVariables(extVars: File): Unit = ???
+
+ override def setExternalVariables(extVars: File, tunable: DaffodilTunables): Unit = ???
+
+ override def setExternalVariables(extVars: Seq[Binding]): Unit = ???
+
+ override def setTunable(tunable: String, value: String): Unit = ???
+
+ override def setTunables(tunables: Map[String, String]): Unit = ???
+
+ /**
+ * Returns an object which contains the result, and/or diagnostic information.
+ */
+ def parse(input: InputStream): ParseResult = {
+ val tempDir = os.temp.dir()
+ val infile = tempDir/"infile"
+ val outfile = tempDir/"outfile"
+ try {
+ os.write(infile, input)
+ val result = os.proc(executableFile, "parse", "-I", "xml", "-o", outfile, infile).call(cwd = tempDir, stderr = os.Pipe)
+ if (result.out.text.isEmpty && result.err.text.isEmpty) {
+ val parseResult = new ParseResult(outfile, Success)
+ parseResult
+ } else {
+ val msg = s"Unexpected daffodil output on stdout: ${result.out.text} on stderr: ${result.err.text}"
+ val parseError = new ParseError(Nope, Nope, Nope, Maybe(msg))
+ val parseResult = new ParseResult(outfile, Failure(parseError))
+ parseResult.addDiagnostic(parseError)
+ parseResult
+ }
+ } catch {
+ case e: os.SubprocessException =>
+ val parseError = if (e.result.out.text.isEmpty && e.result.err.text.isEmpty) {
+ new ParseError(Nope, Nope, Maybe(e), Nope)
+ } else {
+ val msg = s"${e.getMessage} with stdout: ${e.result.out.text} and stderr: ${e.result.err.text}"
+ new ParseError(Nope, Nope, Nope, Maybe(msg))
+ }
+ val parseResult = new ParseResult(outfile, Failure(parseError))
+ parseResult.addDiagnostic(parseError)
+ parseResult
+ }
+ }
+
+ /**
+ * Unparses (that is, serializes) data to the output, returns an object which contains any diagnostics.
+ */
+ def unparse(input: InputStream, output: OutputStream): UnparseResult = {
+ val tempDir = os.temp.dir()
+ val infile = tempDir/"infile"
+ val outfile = tempDir/"outfile"
+ try {
+ os.write(infile, input)
+ val result = os.proc(executableFile, "unparse", "-I", "xml", "-o", outfile, infile).call(cwd = tempDir, stderr = os.Pipe)
+ val finalBitPos0b = os.size(outfile) * 8 // File sizes are bytes, so must multiply to get final position in bits
+ os.read.stream(outfile).writeBytesTo(output)
+ if (result.out.text.isEmpty && result.err.text.isEmpty) {
+ val unparseResult = new UnparseResult(finalBitPos0b, Success)
+ unparseResult
+ } else {
+ val msg = s"Unexpected daffodil output on stdout: ${result.out.text} on stderr: ${result.err.text}"
+ val unparseError = new UnparseError(Nope, Nope, Nope, Maybe(msg))
+ val unparseResult = new UnparseResult(finalBitPos0b, Failure(unparseError))
+ unparseResult.addDiagnostic(unparseError)
+ unparseResult
+ }
+ } catch {
+ case e: os.SubprocessException =>
+ val unparseError = if (e.result.out.text.isEmpty && e.result.err.text.isEmpty) {
+ new UnparseError(Nope, Nope, Maybe(e), Nope)
+ } else {
+ val msg = s"${e.getMessage} with stdout: ${e.result.out.text} and stderr: ${e.result.err.text}"
+ new UnparseError(Nope, Nope, Nope, Maybe(msg))
+ }
+ val finalBitPos0b = 0L
+ val unparseResult = new UnparseResult(finalBitPos0b, Failure(unparseError))
+ unparseResult.addDiagnostic(unparseError)
+ unparseResult
+ }
+ }
+}
+
+object Runtime2DataLocation {
+ class Runtime2DataLocation(_isAtEnd: Boolean,
+ _bitPos1b: Long,
+ _bytePos1b: Long) extends DataLocation {
+ override def isAtEnd: Boolean = _isAtEnd
+ override def bitPos1b: Long = _bitPos1b
+ override def bytePos1b: Long = _bytePos1b
+ }
+
+ def apply(isAtEnd: Boolean = true,
+ bitPos1b: Long = 0L,
+ bytePos1b: Long = 0L): DataLocation = {
+ new Runtime2DataLocation(isAtEnd, bitPos1b, bytePos1b)
+ }
+}
+
+final class ParseResult(outfile: os.Path,
+ override val processorStatus: ProcessorResult,
+ loc: DataLocation = Runtime2DataLocation())
+ extends DFDL.ParseResult
+ with DFDL.State
+ with WithDiagnosticsImpl {
+
+ override def resultState: DFDL.State = this
+
+ override def validationStatus: Boolean = processorStatus.isSuccess
+
+ override def currentLocation: DataLocation = loc
+
+ def infosetAsXML : scala.xml.Elem = {
+ val xml = scala.xml.XML.loadFile(outfile.toIO)
+ xml
+ }
+}
+
+final class UnparseResult(val finalBitPos0b: Long,
+ override val processorStatus: ProcessorResult,
+ loc: DataLocation = Runtime2DataLocation())
+ extends DFDL.UnparseResult
+ with DFDL.State
+ with WithDiagnosticsImpl {
+ /**
+ * Data is 'scannable' if it consists entirely of textual data, and that data
+ * is all in the same encoding.
+ */
+ override def isScannable: Boolean = false // Safest answer since we don't know for sure
+
+ override def encodingName: String = ??? // We don't need encoding unless isScannable is true
+
+ override def validationStatus: Boolean = processorStatus.isSuccess
+
+ override def currentLocation: DataLocation = loc
+
+ override def resultState: DFDL.State = this
+}
diff --git a/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/BinaryIntegerKnownLengthCodeGenerator.scala b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/BinaryIntegerKnownLengthCodeGenerator.scala
new file mode 100644
index 0000000..e1c399d
--- /dev/null
+++ b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/BinaryIntegerKnownLengthCodeGenerator.scala
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.runtime2.generators
+
+import org.apache.daffodil.grammar.primitives.BinaryIntegerKnownLength
+import org.apache.daffodil.schema.annotation.props.gen.{ BitOrder, ByteOrder }
+
+trait BinaryIntegerKnownLengthCodeGenerator {
+
+ def binaryIntegerKnownLengthGenerateCode(g: BinaryIntegerKnownLength, cgState: CodeGeneratorState): Unit = {
+ val e = g.e
+ val fieldName = e.namedQName.local
+ val lengthInBits: Long = {
+ e.schemaDefinitionUnless(e.elementLengthInBitsEv.isConstant, "Runtime dfdl:length expressions not supported.")
+ val len = e.elementLengthInBitsEv.constValue.get
+ len
+ }
+
+ val byteOrder: ByteOrder = {
+ e.schemaDefinitionUnless(e.byteOrderEv.isConstant, "Runtime dfdl:byteOrder expressions not supported.")
+ val bo = e.byteOrderEv.constValue
+ bo
+ }
+ // For the time being this is a very limited back end.
+ // So there are numerous restrictions to enforce.
+ e.schemaDefinitionUnless(lengthInBits <= 64, "Length must be 64 bits or less, but was: %s", lengthInBits)
+ if (lengthInBits == 64 && !g.signed)
+ e.SDE("Integers of 64 bits length must be signed.")
+
+ // This insures we can use regular java.io library calls.
+ if (e.alignmentValueInBits.intValue() % 8 != 0)
+ e.SDE("Only alignment to 8-bit (1 byte) boundaries is supported.")
+
+ // The restrictions below are ones we want to eventually lift.
+ if (lengthInBits != 32)
+ e.SDE("Lengths other than 32 bits are not supported.")
+
+ if (byteOrder ne ByteOrder.BigEndian)
+ e.SDE("Only dfdl:byteOrder 'bigEndian' is supported.")
+
+ if (e.bitOrder ne BitOrder.MostSignificantBitFirst)
+ e.SDE("Only dfdl:bitOrder 'mostSignificantBitFirst' is supported.")
+
+ if (!g.signed)
+ e.SDE("Only signed integers are supported.")
+
+ val initStatement = s" instance->$fieldName = 0xCDCDCDCD;"
+ val parseStatement =
+ s""" if (!error_msg)
+ | {
+ | char buffer[4];
+ | size_t count = fread(&buffer, 1, sizeof(buffer), pstate->stream);
+ | if (count < sizeof(buffer))
+ | {
+ | error_msg = eof_or_error_msg(pstate->stream);
+ | }
+ | instance->$fieldName = be32toh(*((uint32_t *)(&buffer)));
+ | }""".stripMargin
+ val unparseStatement =
+ s""" if (!error_msg)
+ | {
+ | union
+ | {
+ | char c_val[4];
+ | uint32_t i_val;
+ | } buffer;
+ | buffer.i_val = htobe32(instance->$fieldName);
+ | size_t count = fwrite(buffer.c_val, 1, sizeof(buffer), ustate->stream);
+ | if (count < sizeof(buffer))
+ | {
+ | error_msg = eof_or_error_msg(ustate->stream);
+ | }
+ | }""".stripMargin
+ cgState.addSimpleTypeStatements(initStatement, parseStatement, unparseStatement)
+ }
+}
diff --git a/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/CodeGeneratorState.scala b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/CodeGeneratorState.scala
new file mode 100644
index 0000000..bf97636
--- /dev/null
+++ b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/CodeGeneratorState.scala
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.runtime2.generators
+
+import org.apache.daffodil.dpath.NodeInfo
+import org.apache.daffodil.dpath.NodeInfo.PrimType
+import org.apache.daffodil.dsom.ElementBase
+import org.apache.daffodil.exceptions.ThrowsSDE
+
+import scala.collection.mutable
+
+/**
+ * Builds up the state of generated code.
+ */
+class CodeGeneratorState {
+ private val structs = mutable.Stack[ComplexCGState]()
+ private val prototypes = mutable.ArrayBuffer[String]()
+ private val erds = mutable.ArrayBuffer[String]()
+ private val finalStructs = mutable.ArrayBuffer[String]()
+ private val finalImplementation = mutable.ArrayBuffer[String]()
+
+ def addImplementation(context: ElementBase): Unit = {
+ val C = context.namedQName.local
+ val initStatements = structs.top.initStatements.mkString("\n")
+ val parserStatements = structs.top.parserStatements.mkString("\n")
+ val unparserStatements = structs.top.unparserStatements.mkString("\n")
+ val prototypeFunctions =
+ s"""static void ${C}_initSelf($C *instance);
+ |static const char *${C}_parseSelf($C *instance, const PState *pstate);
+ |static const char *${C}_unparseSelf(const $C *instance, const UState *ustate);""".stripMargin
+ prototypes += prototypeFunctions
+ val functions =
+ s"""static void
+ |${C}_initSelf($C *instance)
+ |{
+ |$initStatements
+ |}
+ |
+ |static const char *
+ |${C}_parseSelf($C *instance, const PState *pstate)
+ |{
+ | const char *error_msg = NULL;
+ |$parserStatements
+ | return error_msg;
+ |}
+ |
+ |static const char *
+ |${C}_unparseSelf(const $C *instance, const UState *ustate)
+ |{
+ | const char *error_msg = NULL;
+ |$unparserStatements
+ | return error_msg;
+ |}
+ |""".stripMargin
+ finalImplementation += functions
+ }
+
+ private def defineQNameInit(context: ElementBase): String = {
+ val prefix = context.namedQName.prefix.map(p => s""""$p"""").getOrElse("NULL")
+ val local = context.namedQName.local
+ val nsUri = context.namedQName.namespace.toStringOrNullIfNoNS
+ // Optimize away ns declaration if possible, although this approach may not be entirely correct
+ val parentNsUri = context.enclosingElements.headOption.map(_.namedQName.namespace.toStringOrNullIfNoNS).getOrElse("no-ns")
+ val ns = if (nsUri == null || nsUri == parentNsUri) "NULL" else s""""$nsUri""""
+ val qnameInit =
+ s""" {
+ | $prefix, // namedQName.prefix
+ | "$local", // namedQName.local
+ | $ns, // namedQName.ns
+ | },""".stripMargin
+ qnameInit
+ }
+
+ def addComplexTypeERD(context: ElementBase): Unit = {
+ val C = context.namedQName.local
+ val count = structs.top.declarations.length
+ val offsetComputations = structs.top.offsetComputations.mkString(",\n")
+ val erdComputations = structs.top.erdComputations.mkString(",\n")
+ val qnameInit = defineQNameInit(context)
+ val complexERD =
+ s"""static const $C ${C}_compute_ERD_offsets;
+ |
+ |static const ptrdiff_t ${C}_offsets[$count] = {
+ |$offsetComputations
+ |};
+ |
+ |static const ERD *${C}_childrenERDs[$count] = {
+ |$erdComputations
+ |};
+ |
+ |static const ERD ${C}_ERD = {
+ |$qnameInit
+ | COMPLEX, // typeCode
+ | $count, // numChildren
+ | ${C}_offsets, // offsets
+ | ${C}_childrenERDs, // childrenERDs
+ | (ERDInitSelf)&${C}_initSelf, // initSelf
+ | (ERDParseSelf)&${C}_parseSelf, // parseSelf
+ | (ERDUnparseSelf)&${C}_unparseSelf, // unparseSelf
+ |};
+ |""".stripMargin
+ erds += complexERD
+ }
+
+ def addStruct(context: ElementBase): Unit = {
+ val C = context.namedQName.local
+ val declarations = structs.top.declarations.mkString("\n")
+ val struct =
+ s"""typedef struct $C
+ |{
+ | InfosetBase _base;
+ |$declarations
+ |} $C;
+ |""".stripMargin
+ finalStructs += struct
+ val initStatement = s" instance->_base.erd = &${C}_ERD;"
+ structs.top.initStatements += initStatement
+ }
+
+ def addSimpleTypeStatements(initStatement: String, parseStatement: String, unparseStatement: String): Unit = {
+ structs.top.initStatements += initStatement
+ structs.top.parserStatements += parseStatement
+ structs.top.unparserStatements += unparseStatement
+ }
+
+ def addComplexTypeStatements(child: ElementBase): Unit = {
+ val C = child.namedQName.local
+ val e = child.name
+ val initStatement = s" ${C}_initSelf(&instance->$e);"
+ val parseStatement =
+ s""" if (!error_msg)
+ | {
+ | error_msg = ${C}_parseSelf(&instance->$e, pstate);
+ | }""".stripMargin
+ val unparseStatement =
+ s""" if (!error_msg)
+ | {
+ | error_msg = ${C}_unparseSelf(&instance->$e, ustate);
+ | }""".stripMargin
+ structs.top.initStatements += initStatement
+ structs.top.parserStatements += parseStatement
+ structs.top.unparserStatements += unparseStatement
+ }
+
+ def pushComplexElement(context: ElementBase): Unit = {
+ val C = context.namedQName.local
+ structs.push(new ComplexCGState(C))
+ }
+
+ def popComplexElement(context: ElementBase): Unit = {
+ structs.pop()
+ }
+
+ def addSimpleTypeERD(context: ElementBase): Unit = {
+ val e = context.namedQName.local
+ val qnameInit = defineQNameInit(context)
+ val typeCode = context.optPrimType.get match {
+ case PrimType.Int => "PRIMITIVE_INT32"
+ case PrimType.String => "PRIMITIVE_STRING"
+ case p: PrimType => context.SDE("PrimType %s not supported yet.", p.toString)
+ }
+ val erd =
+ s"""static const ERD ${e}_ERD = {
+ |$qnameInit
+ | $typeCode, // typeCode
+ | 0, // numChildren
+ | NULL, // offsets
+ | NULL, // childrenERDs
+ | NULL, // initSelf
+ | NULL, // parseSelf
+ | NULL, // unparseSelf
+ |};
+ |""".stripMargin
+ erds += erd
+ addComputations(context)
+ }
+
+ def addComputations(child: ElementBase): Unit = {
+ val C = structs.top.C
+ val e = child.namedQName.local
+ val offsetComputation = s" (char *)&${C}_compute_ERD_offsets.$e - (char *)&${C}_compute_ERD_offsets"
+ val erdComputation = s" &${e}_ERD"
+ structs.top.offsetComputations += offsetComputation
+ structs.top.erdComputations += erdComputation
+ }
+
+ def addFieldDeclaration(context: ThrowsSDE, child: ElementBase): Unit = {
+ val definition = if (child.isSimpleType) {
+ import NodeInfo.PrimType
+ child.optPrimType.get match {
+ case PrimType.Long => "int64_t "
+ case PrimType.Int => "int32_t "
+ case x => context.SDE("Unsupported primitive type: " + x)
+ }
+ } else {
+ child.namedQName.local + " "
+ }
+ structs.top.declarations += s" $definition ${child.name};"
+ }
+
+ def generateCodeHeader: String = {
+ val structs = finalStructs.mkString("\n")
+ val header =
+ s"""#ifndef GENERATED_CODE_H
+ |#define GENERATED_CODE_H
+ |
+ |#include "infoset.h" // for InfosetBase
+ |#include <stdint.h> // for int32_t
+
+ |// Define some infoset structures
+ |
+ |$structs
+ |#endif // GENERATED_CODE_H
+ |""".stripMargin
+ header
+ }
+
+ def generateCodeFile(rootElementName: String): String = {
+ val prototypes = this.prototypes.mkString("\n")
+ val erds = this.erds.mkString("\n")
+ val finalImplementation = this.finalImplementation.mkString("\n")
+ val code =
+ s"""#include "generated_code.h" // for generated code structs
+ |#include <endian.h> // for be32toh, htobe32
+ |#include <stddef.h> // for ptrdiff_t
+ |#include <stdio.h> // for NULL, fread, fwrite, size_t, FILE
+ |
+ |// Prototypes needed for compilation
+ |
+ |$prototypes
+ |
+ |// Metadata singletons
+ |
+ |$erds
+ |// Return a root element to be used for parsing or unparsing
+ |
+ |InfosetBase *
+ |rootElement()
+ |{
+ | static $rootElementName instance;
+ | InfosetBase *root = &instance._base;
+ | ${rootElementName}_ERD.initSelf(root);
+ | return root;
+ |}
+ |
+ |// Methods to initialize, parse, and unparse infoset nodes
+ |
+ |$finalImplementation
+ |""".stripMargin
+ code
+ }
+}
+
+/**
+ * Accumulates strings of generated C code for nested elements inside
+ * complex elements.
+ */
+class ComplexCGState(val C: String) {
+ val declarations = mutable.ArrayBuffer[String]()
+ val offsetComputations = mutable.ArrayBuffer[String]()
+ val erdComputations = mutable.ArrayBuffer[String]()
+ val initStatements = mutable.ArrayBuffer[String]()
+ val parserStatements = mutable.ArrayBuffer[String]()
+ val unparserStatements = mutable.ArrayBuffer[String]()
+}
diff --git a/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/ElementParseAndUnspecifiedLengthCodeGenerator.scala b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/ElementParseAndUnspecifiedLengthCodeGenerator.scala
new file mode 100644
index 0000000..d13be31
--- /dev/null
+++ b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/ElementParseAndUnspecifiedLengthCodeGenerator.scala
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.runtime2.generators
+
+import org.apache.daffodil.grammar.primitives.ElementParseAndUnspecifiedLength
+import org.apache.daffodil.runtime2.Runtime2CodeGenerator
+
+trait ElementParseAndUnspecifiedLengthCodeGenerator {
+
+ def elementParseAndUnspecifiedLengthGenerateCode(g: ElementParseAndUnspecifiedLength,
+ cgState: CodeGeneratorState): Unit = {
+ val context = g.context
+ val elementContentGram = g.eGram // a Gram isA ParserGenerator
+
+ context.schemaDefinitionWhen(context.inputValueCalcOption.isDefined, "Elements with inputValueCalc are not supported.")
+ context.schemaDefinitionWhen(context.outputValueCalcOption.isDefined, "Elements with outputValueCalc are not supported.")
+ context.schemaDefinitionUnless(g.eBeforeGram.isEmpty, "Statements associated with elements are not supported.")
+ context.schemaDefinitionUnless(g.eAfterGram.isEmpty, "Statements associated with elements are not supported.")
+ context.schemaDefinitionUnless(g.repTypeElementGram.isEmpty, "dfdlx:repType is not supported.")
+
+ if (context.isSimpleType) {
+ cgState.addSimpleTypeERD(context) // ERD static initializer
+ Runtime2CodeGenerator.generateCode(elementContentGram, cgState) // initSelf, parseSelf, unparseSelf
+ } else {
+ cgState.pushComplexElement(context)
+ context.elementChildren.foreach { child =>
+ if (!child.isSimpleType) {
+ cgState.addComplexTypeStatements(child) // recursive calls to parse, unparse, init
+ cgState.addComputations(child) // offset, ERD computations
+ }
+ cgState.addFieldDeclaration(context, child) // struct member for child
+ Runtime2CodeGenerator.generateCode(child.enclosedElement, cgState) // generate children too
+ }
+ cgState.addStruct(context) // struct definition
+ cgState.addImplementation(context) // initSelf, parseSelf, unparseSelf
+ cgState.addComplexTypeERD(context) // ERD static initializer
+ cgState.popComplexElement(context)
+ }
+ }
+}
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/OrderedSequenceCodeGenerator.scala
similarity index 55%
copy from daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
copy to daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/OrderedSequenceCodeGenerator.scala
index f0bc051..fe1ee33 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
+++ b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/OrderedSequenceCodeGenerator.scala
@@ -15,20 +15,20 @@
* limitations under the License.
*/
-package org.apache.daffodil.grammar
+package org.apache.daffodil.runtime2.generators
-import org.apache.daffodil.grammar.primitives.UnicodeByteOrderMark
-import org.apache.daffodil.dsom.Root
+import org.apache.daffodil.grammar.primitives.OrderedSequence
+import org.apache.daffodil.runtime2.Runtime2CodeGenerator
-trait RootGrammarMixin
- extends LocalElementGrammarMixin // can be repeating if not root
- { self: Root =>
+trait OrderedSequenceCodeGenerator {
- final lazy val document = prod("document") {
- schemaDefinitionUnless(isScalar, "The document element cannot be an array.")
- UnicodeByteOrderMark(this) ~ documentElement
+ def orderedSequenceGenerateCode(g: OrderedSequence, cgState: CodeGeneratorState): Unit = {
+ //
+ // To lift this draconian restriction, we have to
+ // generate code for each of the children, and combine them into a block
+ //
+ g.schemaDefinitionUnless(g.sequenceChildren.length == 1, "Only a single child of a sequence is supported.")
+ val child1 = g.sequenceChildren(0)
+ Runtime2CodeGenerator.generateCode(child1, cgState)
}
-
- private def documentElement = enclosedElement
-
}
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/SeqCompCodeGenerator.scala
similarity index 63%
copy from daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
copy to daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/SeqCompCodeGenerator.scala
index f0bc051..2954fdb 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
+++ b/daffodil-runtime2/src/main/scala/org/apache/daffodil/runtime2/generators/SeqCompCodeGenerator.scala
@@ -15,20 +15,16 @@
* limitations under the License.
*/
-package org.apache.daffodil.grammar
+package org.apache.daffodil.runtime2.generators
-import org.apache.daffodil.grammar.primitives.UnicodeByteOrderMark
-import org.apache.daffodil.dsom.Root
+import org.apache.daffodil.grammar.SeqComp
+import org.apache.daffodil.runtime2.Runtime2CodeGenerator
-trait RootGrammarMixin
- extends LocalElementGrammarMixin // can be repeating if not root
- { self: Root =>
+trait SeqCompCodeGenerator {
- final lazy val document = prod("document") {
- schemaDefinitionUnless(isScalar, "The document element cannot be an array.")
- UnicodeByteOrderMark(this) ~ documentElement
+ def seqCompGenerateCode(g: SeqComp, state: CodeGeneratorState): Unit = {
+ for (elem <- g.children) {
+ Runtime2CodeGenerator.generateCode(elem, state)
+ }
}
-
- private def documentElement = enclosedElement
-
}
diff --git a/daffodil-runtime2/src/main/scala/org/apache/daffodil/tdml/processor/runtime2/Runtime2TDMLDFDLProcessor.scala b/daffodil-runtime2/src/main/scala/org/apache/daffodil/tdml/processor/runtime2/Runtime2TDMLDFDLProcessor.scala
new file mode 100644
index 0000000..97e2e4b
--- /dev/null
+++ b/daffodil-runtime2/src/main/scala/org/apache/daffodil/tdml/processor/runtime2/Runtime2TDMLDFDLProcessor.scala
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.tdml.processor.runtime2
+
+import dev.dirs.ProjectDirectories
+import org.apache.daffodil.api._
+import org.apache.daffodil.compiler.Compiler
+import org.apache.daffodil.externalvars.Binding
+import org.apache.daffodil.runtime2.ParseResult
+import org.apache.daffodil.runtime2.Runtime2DataProcessor
+import org.apache.daffodil.runtime2.UnparseResult
+import org.apache.daffodil.tdml.processor._
+import org.apache.daffodil.xml.QName
+import org.apache.daffodil.xml.XMLUtils
+
+import scala.xml.Node
+
+final class TDMLDFDLProcessorFactory private(
+ private var compiler: Compiler,
+ private var checkAllTopLevel: Boolean,
+ validateDFDLSchemasArg: Boolean)
+ extends AbstractTDMLDFDLProcessorFactory {
+
+ override def validateDFDLSchemas = validateDFDLSchemasArg
+
+ override type R = TDMLDFDLProcessorFactory
+
+ override def implementationName = "daffodil-runtime2"
+
+ def this() = this(compiler = Compiler(validateDFDLSchemas = true),
+ checkAllTopLevel = false,
+ validateDFDLSchemasArg = true)
+
+ private def copy(
+ compiler: Compiler = compiler,
+ checkAllTopLevel: Boolean = checkAllTopLevel,
+ validateDFDLSchemas: Boolean = validateDFDLSchemas) =
+ new TDMLDFDLProcessorFactory(compiler, checkAllTopLevel, validateDFDLSchemas)
+
+ /**
+ * Deprecated methods must be implemented. Some are just stubs though now.
+ */
+ @deprecated("Use withValidateDFDLSchemas.", "2.6.0")
+ override def setValidateDFDLSchemas(bool: Boolean): Unit = {
+ compiler = compiler.withValidateDFDLSchemas(bool)
+ }
+
+ override def withValidateDFDLSchemas(bool: Boolean): TDMLDFDLProcessorFactory = {
+ copy(compiler = compiler.withValidateDFDLSchemas(bool))
+ }
+
+ @deprecated("Use withCheckAllTopLevel.", "2.6.0")
+ override def setCheckAllTopLevel(checkAllTopLevel: Boolean): Unit = {
+ compiler = compiler.withCheckAllTopLevel(checkAllTopLevel)
+ }
+
+ override def withCheckAllTopLevel(checkAllTopLevel: Boolean): TDMLDFDLProcessorFactory = {
+ copy(compiler = compiler.withCheckAllTopLevel(checkAllTopLevel))
+ }
+
+ @deprecated("Use withTunables.", "2.6.0")
+ override def setTunables(tunables: Map[String, String]): Unit =
+ compiler = compiler.withTunables(tunables)
+
+ override def withTunables(tunables: Map[String, String]): TDMLDFDLProcessorFactory =
+ copy(compiler = compiler.withTunables(tunables))
+
+ @deprecated("Use DaffodilTDMLDFDLProcessor.setExternalDFDLVariables.", "2.6.0")
+ override def setExternalDFDLVariables(externalVarBindings: Seq[Binding]): Unit =
+ compiler = compiler.withExternalDFDLVariablesImpl(externalVarBindings)
+
+ override def withExternalDFDLVariables(externalVarBindings: Seq[Binding]): TDMLDFDLProcessorFactory =
+ copy(compiler = compiler.withExternalDFDLVariablesImpl(externalVarBindings))
+
+ @deprecated("Use arguments to getProcessor()", "2.6.0")
+ override def setDistinguishedRootNode(name: String, namespace: String): Unit =
+ compiler = compiler.withDistinguishedRootNode(name, namespace)
+
+ // Return result is a TDML.CompileResult - so it's the result
+ // of compiling the schema for the test.
+ override def getProcessor(
+ schemaSource: DaffodilSchemaSource,
+ useSerializedProcessor: Boolean,
+ optRootName: Option[String] = None,
+ optRootNamespace: Option[String] = None): TDML.CompileResult = {
+
+ // Compile the DFDL schema into a ProcessorFactory
+ val pf = compiler.compileSource(schemaSource, optRootName, optRootNamespace)
+ val res = if (pf.isError) {
+ Left(pf.getDiagnostics) // DFDL schema compilation diagnostics
+ } else {
+ // Create a CodeGenerator from the DFDL schema for the C language
+ val generator = pf.forLanguage("c")
+
+ // Generate the C source code in our cache directory
+ val rootNS = QName.refQNameFromExtendedSyntax(optRootName.getOrElse("")).toOption
+ val directories = ProjectDirectories.from("org", "Apache Software Foundation", "Daffodil")
+ val outputDir = generator.generateCode(rootNS, directories.cacheDir)
+
+ // Compile the generated code into an executable
+ val executable = generator.compileCode(outputDir)
+
+ // Summarize the result of compiling the schema for the test
+ val compileResult = if (generator.isError) {
+ Left(generator.getDiagnostics) // C code compilation diagnostics
+ } else {
+ // Create a processor for running the test using the executable
+ val processor = new Runtime2TDMLDFDLProcessor(executable)
+ Right((generator.getDiagnostics, processor))
+ }
+ compileResult
+ }
+ res
+ }
+
+}
+
+/**
+ * Delegates all execution, error gathering, error access to the [[Runtime2DataProcessor]].
+ * The responsibility of this class is just for TDML matching up. That is dealing with TDML
+ * XML Infosets, feeding to the unparser, creating XML from the result created by the
+ * [[Runtime2DataProcessor]]. All the "real work" is done by [[Runtime2DataProcessor]].
+ */
+class Runtime2TDMLDFDLProcessor(executable: os.Path) extends TDMLDFDLProcessor {
+
+ override type R = Runtime2TDMLDFDLProcessor
+
+ private val dataProcessor = new Runtime2DataProcessor(executable)
+ private var anyErrors: Boolean = false
+ private var diagnostics: Seq[Diagnostic] = Nil
+
+ @deprecated("Use withDebugging.", "2.6.0")
+ override def setDebugging(b: Boolean) = ???
+ override def withDebugging(b: Boolean): Runtime2TDMLDFDLProcessor = this
+
+ @deprecated("Use withTracing.", "2.6.0")
+ override def setTracing(bool: Boolean): Unit = ???
+ override def withTracing(bool: Boolean): Runtime2TDMLDFDLProcessor = this
+
+ @deprecated("Use withDebugger.", "2.6.0")
+ override def setDebugger(db: AnyRef): Unit = ???
+ override def withDebugger(db: AnyRef): Runtime2TDMLDFDLProcessor = this
+
+ @deprecated("Use withValidationMode.", "2.6.0")
+ override def setValidationMode(validationMode: ValidationMode.Type): Unit = ???
+ override def withValidationMode(validationMode: ValidationMode.Type): Runtime2TDMLDFDLProcessor = this
+
+ @deprecated("Use withExternalDFDLVariables.", "2.6.0")
+ override def setExternalDFDLVariables(externalVarBindings: Seq[Binding]): Unit = ???
+ override def withExternalDFDLVariables(externalVarBindings: Seq[Binding]): Runtime2TDMLDFDLProcessor = this
+
+ // Save any errors from running the C code here to be returned later
+ override def isError: Boolean = anyErrors
+ override def getDiagnostics: Seq[Diagnostic] = diagnostics
+
+ // Run the C code, collect and save the infoset with any errors and
+ // diagnostics, and return a [[TDMLParseResult]] summarizing the result.
+ // The C code will run in a subprocess, parse the input stream, write
+ // an XML infoset on its standard output, and write any error messages
+ // on its standard output (all done in [[Runtime2DataProcessor.parse]]).
+ override def parse(is: java.io.InputStream, lengthLimitInBits: Long): TDMLParseResult = {
+ // TODO: pass lengthLimitInBits to the C program to tell it how big the data is
+ val pr = dataProcessor.parse(is)
+ anyErrors = pr.isError
+ diagnostics = pr.getDiagnostics
+ new Runtime2TDMLParseResult(pr)
+ }
+
+ // Run the C code, collect and save the unparsed data with any errors and
+ // diagnostics, and return a [[TDMLUnparseResult]] summarizing the result.
+ // The C code will run in a subprocess, unparse the input stream, write
+ // the unparsed data on its standard output, and write any error messages
+ // on its standard output (all done in [[Runtime2DataProcessor.unparse]]).
+ override def unparse(infosetXML: scala.xml.Node, outStream: java.io.OutputStream): TDMLUnparseResult = {
+ val tempDir = null
+ val tempInputFile = XMLUtils.convertNodeToTempFile(infosetXML, tempDir)
+ val inStream = os.read.inputStream(os.Path(tempInputFile))
+ val upr = dataProcessor.unparse(inStream, outStream)
+ anyErrors = upr.isError
+ diagnostics = upr.getDiagnostics
+ new Runtime2TDMLUnparseResult(upr)
+ }
+
+ def unparse(parseResult: TDMLParseResult, outStream: java.io.OutputStream): TDMLUnparseResult = {
+ unparse(parseResult.getResult, outStream)
+ }
+}
+
+final class Runtime2TDMLParseResult(pr: ParseResult) extends TDMLParseResult {
+ override def addDiagnostic(failure: Diagnostic): Unit = pr.addDiagnostic(failure)
+
+ override def getResult: Node = pr.infosetAsXML
+
+ override def currentLocation: DataLocation = pr.currentLocation
+
+ override def isValidationError: Boolean = pr.isValidationError
+
+ override def isProcessingError: Boolean = pr.isProcessingError
+
+ override def getDiagnostics: Seq[Diagnostic] = pr.getDiagnostics
+}
+
+final class Runtime2TDMLUnparseResult(upr: UnparseResult) extends TDMLUnparseResult {
+ override def bitPos0b: Long = upr.finalBitPos0b
+
+ override def finalBitPos0b: Long = upr.finalBitPos0b
+
+ override def isScannable: Boolean = upr.isScannable
+
+ override def encodingName: String = upr.encodingName
+
+ override def isValidationError: Boolean = upr.isValidationError
+
+ override def isProcessingError: Boolean = upr.isProcessingError
+
+ override def getDiagnostics: Seq[Diagnostic] = upr.getDiagnostics
+}
diff --git a/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/TestRuntime2.dfdl.xsd b/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/TestRuntime2.dfdl.xsd
new file mode 100644
index 0000000..81cade0
--- /dev/null
+++ b/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/TestRuntime2.dfdl.xsd
@@ -0,0 +1,50 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<xs:schema elementFormDefault="qualified"
+ targetNamespace="http://example.com"
+ xmlns="http://example.com"
+ xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/"
+ xmlns:ex="http://example.com"
+ xmlns:xs="http://www.w3.org/2001/XMLSchema">
+
+ <xs:include schemaLocation="org/apache/daffodil/xsd/DFDLGeneralFormat.dfdl.xsd"/>
+
+ <xs:annotation>
+ <xs:appinfo source="http://www.ogf.org/dfdl/">
+ <dfdl:format ref="GeneralFormat" representation="binary"/>
+ </xs:appinfo>
+ </xs:annotation>
+
+ <xs:element name="ex_int32">
+ <xs:complexType>
+ <xs:sequence>
+ <xs:element name="e1" type="xs:int"/>
+ <xs:element name="c2">
+ <xs:complexType>
+ <xs:sequence>
+ <xs:element name="e2" type="xs:int"/>
+ <xs:element name="e3" type="xs:int"/>
+ </xs:sequence>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ </xs:complexType>
+ </xs:element>
+
+</xs:schema>
diff --git a/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/TestRuntime2.tdml b/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/TestRuntime2.tdml
new file mode 100644
index 0000000..7a5d08b
--- /dev/null
+++ b/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/TestRuntime2.tdml
@@ -0,0 +1,67 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+<tdml:testSuite
+ defaultConfig="config-runtime2"
+ defaultImplementations="daffodil daffodil-runtime2"
+ defaultRoundTrip="onePass"
+ description="TDML tests for runtime2"
+ suiteName="TestRuntime2"
+ xmlns:daf="urn:ogf:dfdl:2013:imp:daffodil.apache.org:2018:ext"
+ xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/"
+ xmlns:ex="http://example.com"
+ xmlns:tdml="http://www.ibm.com/xmlns/dfdl/testData"
+ xmlns="http://example.com">
+
+ <tdml:defineConfig name="config-runtime1">
+ <daf:tunables>
+ <daf:tdmlImplementation>daffodil</daf:tdmlImplementation>
+ </daf:tunables>
+ </tdml:defineConfig>
+
+ <tdml:defineConfig name="config-runtime2">
+ <daf:tunables>
+ <daf:tdmlImplementation>daffodil-runtime2</daf:tdmlImplementation>
+ </daf:tunables>
+ </tdml:defineConfig>
+
+ <tdml:parserTestCase
+ description="Parse binary int32 integers"
+ model="TestRuntime2.dfdl.xsd"
+ name="parse_int32">
+ <tdml:document>
+ <tdml:documentPart type="file">parse_int32</tdml:documentPart>
+ </tdml:document>
+ <tdml:infoset>
+ <tdml:dfdlInfoset type="file">unparse_int32</tdml:dfdlInfoset>
+ </tdml:infoset>
+ </tdml:parserTestCase>
+
+ <tdml:unparserTestCase
+ description="Unparse binary int32 numbers"
+ model="TestRuntime2.dfdl.xsd"
+ name="unparse_int32">
+ <tdml:infoset>
+ <tdml:dfdlInfoset type="file">unparse_int32</tdml:dfdlInfoset>
+ </tdml:infoset>
+ <tdml:document>
+ <tdml:documentPart type="file">parse_int32</tdml:documentPart>
+ </tdml:document>
+ </tdml:unparserTestCase>
+
+</tdml:testSuite>
diff --git a/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/parse_int32 b/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/parse_int32
new file mode 100644
index 0000000..fce373f
Binary files /dev/null and b/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/parse_int32 differ
diff --git a/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/unparse_int32 b/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/unparse_int32
new file mode 100644
index 0000000..ea15640
--- /dev/null
+++ b/daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/unparse_int32
@@ -0,0 +1,25 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<ex:ex_int32 xmlns:ex="http://example.com">
+ <ex:e1>1</ex:e1>
+ <ex:c2>
+ <ex:e2>2</ex:e2>
+ <ex:e3>3</ex:e3>
+ </ex:c2>
+</ex:ex_int32>
diff --git a/daffodil-runtime2/src/test/scala/org/apache/daffodil/runtime2/TestCodeGenerator.scala b/daffodil-runtime2/src/test/scala/org/apache/daffodil/runtime2/TestCodeGenerator.scala
new file mode 100644
index 0000000..19d5682
--- /dev/null
+++ b/daffodil-runtime2/src/test/scala/org/apache/daffodil/runtime2/TestCodeGenerator.scala
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.daffodil.runtime2
+
+import java.io.ByteArrayInputStream
+import java.io.ByteArrayOutputStream
+import java.nio.channels.Channels
+
+import org.apache.daffodil.Implicits.intercept
+import org.apache.daffodil.compiler.Compiler
+import org.apache.daffodil.util.Misc
+import org.apache.daffodil.util.SchemaUtils
+import org.apache.daffodil.util.TestUtils
+import org.junit.Assert.assertArrayEquals
+import org.junit.Test
+
+/**
+ * Checks that we can create a [[CodeGenerator]] and call its methods.
+ * The value of this test is to debug the call path from [[Compiler]]
+ * to [[CodeGenerator]] for one very simple DFDL schema. Running TDML
+ * tests with daffodil-runtime2 is a more effective way to test the
+ * functionality of CodeGenerator's generated code for as many DFDL
+ * schemas as you could want.
+ */
+class TestCodeGenerator {
+ // Define a simple DFDL test schema for debugging our code path
+ private val testSchema = SchemaUtils.dfdlTestSchema(
+ <xs:include schemaLocation="org/apache/daffodil/xsd/DFDLGeneralFormat.dfdl.xsd"/>,
+ <dfdl:format representation="binary" ref="GeneralFormat"/>,
+ <xs:element name="e1">
+ <xs:complexType>
+ <xs:sequence>
+ <xs:element name="x" type="xs:int"/>
+ </xs:sequence>
+ </xs:complexType>
+ </xs:element>)
+
+ @Test def test_forLanguageC_success(): Unit = {
+ // Create a ProcessorFactory from the test schema
+ val pf = Compiler().compileNode(testSchema)
+ assert(!pf.isError, pf.getDiagnostics.map(_.getMessage()).mkString("\n"))
+
+ // Create a CodeGenerator from the ProcessorFactory for a supported language
+ val cg = pf.forLanguage("c")
+ assert(!cg.isError, cg.getDiagnostics.map(_.getMessage()).mkString("\n"))
+ }
+
+ @Test def test_forLanguage_error(): Unit = {
+ // Create a ProcessorFactory from the test schema
+ val pf = Compiler().compileNode(testSchema)
+ assert(!pf.isError, pf.getDiagnostics.map(_.getMessage()).mkString("\n"))
+
+ // Create a CodeGenerator from the ProcessorFactory for an unsupported language
+ val e = intercept[Exception] {
+ pf.forLanguage("vhld")
+ }
+ assert(e.getMessage.contains("source language vhld is not supported"))
+ }
+
+ @Test def test_generateCode_success(): Unit = {
+ // Create a ProcessorFactory and CodeGenerator from the test schema
+ val pf = Compiler().compileNode(testSchema)
+ val cg = pf.forLanguage("c")
+
+ // Generate code from the test schema successfully
+ val outputDir = cg.generateCode(None, "./generateCode_tmp")
+ assert(!cg.isError, cg.getDiagnostics.map(_.getMessage()).mkString("\n"))
+ assert(os.exists(outputDir))
+ assert(os.exists(outputDir/"c"/"generated_code.c"))
+
+ // Remove the generated code
+ os.remove.all(outputDir)
+ }
+
+ @Test def test_compileCode_success(): Unit = {
+ // Create a CodeGenerator and generate code from the test schema
+ val pf = Compiler().compileNode(testSchema)
+ val cg = pf.forLanguage("c")
+ val outputDir = cg.generateCode(None, "./compileCode_tmp")
+
+ // Compile the generated code into an executable successfully
+ val executable = cg.compileCode(outputDir)
+ assert(!cg.isError, cg.getDiagnostics.map(_.getMessage()).mkString("\n"))
+ assert(os.exists(executable))
+
+ // Remove the generated code
+ os.remove.all(outputDir)
+ }
+
+ @Test def test_parse_success(): Unit = {
+ // Compile the test schema into a C executable
+ val pf = Compiler().compileNode(testSchema)
+ val cg = pf.forLanguage("c")
+ val outputDir = cg.generateCode(None, "./parse_tmp")
+ val executable = cg.compileCode(outputDir)
+
+ // Create a Runtime2DataProcessor and parse a binary int32 number successfully
+ val dp = new Runtime2DataProcessor(executable)
+ val b = Misc.hex2Bytes("00000005")
+ val input = new ByteArrayInputStream(b)
+ val pr = dp.parse(input)
+ assert(!pr.isError && pf.getDiagnostics.isEmpty, pr.getDiagnostics.map(_.getMessage()).mkString("\n"))
+ val expected = <e1><x>5</x></e1>
+ TestUtils.assertEqualsXMLElements(expected, pr.infosetAsXML)
+
+ // Remove the generated code
+ os.remove.all(outputDir)
+ }
+
+ @Test def test_unparse_success(): Unit = {
+ // Compile the test schema into a C executable
+ val pf = Compiler().compileNode(testSchema)
+ val cg = pf.forLanguage("c")
+ val outputDir = cg.generateCode(None, "./unparse_tmp")
+ val executable = cg.compileCode(outputDir)
+
+ // Create a Runtime2DataProcessor and unparse a binary int32 number successfully
+ val dp = new Runtime2DataProcessor(executable)
+ val input = Channels.newInputStream(Misc.stringToReadableByteChannel("<e1><x>5</x></e1>"))
+ val output = new ByteArrayOutputStream()
+ val pr = dp.unparse(input, output)
+ assert(!pr.isError && pf.getDiagnostics.isEmpty, pr.getDiagnostics.map(_.getMessage()).mkString("\n"))
+ val expected = Misc.hex2Bytes("00000005")
+ assertArrayEquals(expected, output.toByteArray)
+
+ // Remove the generated code
+ os.remove.all(outputDir)
+ }
+}
diff --git a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala b/daffodil-runtime2/src/test/scala/org/apache/daffodil/runtime2/TestRuntime2.scala
similarity index 61%
copy from daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
copy to daffodil-runtime2/src/test/scala/org/apache/daffodil/runtime2/TestRuntime2.scala
index f0bc051..9bada7c 100644
--- a/daffodil-core/src/main/scala/org/apache/daffodil/grammar/ElementDeclGrammarMixin.scala
+++ b/daffodil-runtime2/src/test/scala/org/apache/daffodil/runtime2/TestRuntime2.scala
@@ -15,20 +15,22 @@
* limitations under the License.
*/
-package org.apache.daffodil.grammar
+package org.apache.daffodil.runtime2
-import org.apache.daffodil.grammar.primitives.UnicodeByteOrderMark
-import org.apache.daffodil.dsom.Root
+import org.junit.Test
+import org.apache.daffodil.tdml.Runner
+import org.junit.AfterClass
-trait RootGrammarMixin
- extends LocalElementGrammarMixin // can be repeating if not root
- { self: Root =>
+object TestRuntime2 {
+ val testDir = "/org/apache/daffodil/runtime2/"
+ val runner = Runner(testDir, "TestRuntime2.tdml")
- final lazy val document = prod("document") {
- schemaDefinitionUnless(isScalar, "The document element cannot be an array.")
- UnicodeByteOrderMark(this) ~ documentElement
- }
+ @AfterClass def shutDown(): Unit = { runner.reset }
+}
- private def documentElement = enclosedElement
+class TestRuntime2 {
+ import TestRuntime2._
+ @Test def test_parse_int32(): Unit = { runner.runOneTest("parse_int32") }
+ @Test def test_unparse_int32(): Unit = { runner.runOneTest("unparse_int32") }
}
diff --git a/daffodil-tdml-lib/src/main/scala/org/apache/daffodil/tdml/TDMLRunner.scala b/daffodil-tdml-lib/src/main/scala/org/apache/daffodil/tdml/TDMLRunner.scala
index 0ac8fa6..a337045 100644
--- a/daffodil-tdml-lib/src/main/scala/org/apache/daffodil/tdml/TDMLRunner.scala
+++ b/daffodil-tdml-lib/src/main/scala/org/apache/daffodil/tdml/TDMLRunner.scala
@@ -477,7 +477,12 @@ abstract class TestCase(testCaseXML: NodeSeq, val parent: DFDLTestSuite)
lazy val tdmlDFDLProcessorFactory: AbstractTDMLDFDLProcessorFactory = {
import scala.language.existentials
- val className = "org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory"
+ // Our TDMLDFDLProcessorFactory implementation is a tunable choice with three values.
+ val className = tunableObj.tdmlImplementation match {
+ case "daffodil" => "org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory"
+ case "daffodil-runtime2" => "org.apache.daffodil.tdml.processor.runtime2.TDMLDFDLProcessorFactory"
+ case "ibm" => "org.apache.daffodil.tdml.processor.ibm.TDMLDFDLProcessorFactory"
+ }
//
// If you haven't seen it before. Check out this Try(...) idiom.
@@ -1334,7 +1339,7 @@ case class UnparserTestCase(ptc: NodeSeq, parentArg: DFDLTestSuite)
if (testDataLength >= 0) {
val fullBytesNeeded = (testDataLength + 7) / 8
if (testData.length != fullBytesNeeded) {
- throw TDMLException("Unparse result data was was %d bytes, but the result length (%d bits) requires %d bytes.".format(
+ throw TDMLException("Unparse result data was %d bytes, but the result length (%d bits) requires %d bytes.".format(
testData.length, testDataLength, fullBytesNeeded), implString)
}
}
diff --git a/project/Dependencies.scala b/project/Dependencies.scala
index fd076c7..5dbf7e3 100644
--- a/project/Dependencies.scala
+++ b/project/Dependencies.scala
@@ -29,7 +29,9 @@ object Dependencies {
"xml-resolver" % "xml-resolver" % "1.2",
"commons-io" % "commons-io" % "2.8.0",
"jline" % "jline" % "2.14.6",
- "com.typesafe" % "config" % "1.4.0"
+ "com.typesafe" % "config" % "1.4.0",
+ "com.lihaoyi" %% "os-lib" % "0.7.1", // for generating C source files
+ "dev.dirs" % "directories" % "21" // for caching compiled C files
)
lazy val infoset = Seq(
@@ -37,7 +39,7 @@ object Dependencies {
"com.fasterxml.woodstox" % "woodstox-core" % "6.2.3",
"com.fasterxml.jackson.core" % "jackson-core" % "2.11.3"
)
-
+
lazy val cli = Seq(
"org.fusesource.jansi" % "jansi" % "1.17.1",
"org.rogach" %% "scallop" % "3.5.1",
diff --git a/project/Rat.scala b/project/Rat.scala
index 2489a69..062dadb 100644
--- a/project/Rat.scala
+++ b/project/Rat.scala
@@ -114,6 +114,7 @@ object Rat {
file("daffodil-sapi/src/test/resources/test/sapi/myData16.dat"),
file("daffodil-sapi/src/test/resources/test/sapi/myData19.dat"),
file("daffodil-sapi/src/test/resources/test/sapi/myDataBroken.dat"),
+ file("daffodil-runtime2/src/test/resources/org/apache/daffodil/runtime2/parse_int32"),
file("daffodil-tdml-lib/src/test/resources/test/tdml/test.bin"),
file("daffodil-tdml-lib/src/test/resources/test/tdml/test.txt"),
file("daffodil-tdml-processor/src/test/resources/test/tdml/test.bin"),
diff --git a/project/plugins.sbt b/project/plugins.sbt
index 6918bca..b936781 100644
--- a/project/plugins.sbt
+++ b/project/plugins.sbt
@@ -15,6 +15,8 @@
* limitations under the License.
*/
+addSbtPlugin("com.github.tnakamot" % "sbt-cc" % "0.0.3")
+
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.7.5")
addSbtPlugin("org.musigma" % "sbt-rat" % "0.7.0")