You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by gi...@apache.org on 2023/02/23 20:39:18 UTC
[daffodil-site] branch asf-site updated: Publishing from 4c1bfdaafb8f0a17942383886dadb1e7743dfa34
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/daffodil-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 2fd796d Publishing from 4c1bfdaafb8f0a17942383886dadb1e7743dfa34
2fd796d is described below
commit 2fd796d40aa4718f48cd7ed57c191349c461f25d
Author: Apache Daffodil Site Autobuild <de...@daffodil.apache.org>
AuthorDate: Thu Feb 23 20:39:13 2023 +0000
Publishing from 4c1bfdaafb8f0a17942383886dadb1e7743dfa34
---
content/cli/index.html | 2 +-
.../{runtime2-todos => daffodilc-todos}/index.html | 101 ++++++---------------
content/releases/3.1.0/index.html | 13 +--
content/releases/3.2.0/index.html | 6 +-
content/releases/3.2.1/index.html | 2 +-
content/releases/3.3.0/index.html | 2 +-
content/releases/3.4.0/index.html | 2 +-
7 files changed, 44 insertions(+), 84 deletions(-)
diff --git a/content/cli/index.html b/content/cli/index.html
index dc6af71..fb338d5 100644
--- a/content/cli/index.html
+++ b/content/cli/index.html
@@ -127,7 +127,7 @@
</dd>
<dt><code class="language-plaintext highlighter-rouge">CC</code></dt>
<dd>
- <p>If you need to specify which C compiler should be called when running TDML files with the <code class="language-plaintext highlighter-rouge">test</code> subcommand using Daffodil's runtime2 backend, you can set the <code class="language-plaintext highlighter-rouge">CC</code> environment variable. If that environment variable is not specified, then Daffodil will call the first C compiler driver command it finds within the <code class="language-plaintext highlighter-rouge">PATH</cod [...]
+ <p>If you need to specify which C compiler should be called when running TDML files with the <code class="language-plaintext highlighter-rouge">test</code> subcommand using Daffodil's codegen-c backend, you can set the <code class="language-plaintext highlighter-rouge">CC</code> environment variable. If that environment variable is not specified, then Daffodil will call the first C compiler driver command it finds within the <code class="language-plaintext highlighter-rouge">PATH</co [...]
</dd>
</dl>
diff --git a/content/dev/design-notes/runtime2-todos/index.html b/content/dev/design-notes/daffodilc-todos/index.html
similarity index 83%
rename from content/dev/design-notes/runtime2-todos/index.html
rename to content/dev/design-notes/daffodilc-todos/index.html
index fa86670..831b66e 100644
--- a/content/dev/design-notes/runtime2-todos/index.html
+++ b/content/dev/design-notes/daffodilc-todos/index.html
@@ -86,13 +86,13 @@
<div class="row">
<div class="col-md-12">
<div class="sect1">
-<h2 id="runtime2-todos">Runtime2 ToDos</h2>
+<h2 id="c-code-generator-todos">C Code Generator ToDos</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="overview">Overview</h3>
<div class="paragraph">
-<p>We have built an initial DFDL-to-C backend
-and code generator for Apache Daffodil.
+<p>We have built an initial DFDL-to-C code
+generator and backend for Apache Daffodil.
Currently the C code generator can support
binary boolean, integer, and real numbers,
arrays of simple and complex elements,
@@ -178,7 +178,7 @@ doesn’t integrate well with any of these systems
unless we generate a child element with a generated name
(which makes paths awkward, etc.).
Hence, it seems better to say that
-the runtime2 DFDL subset doesn’t allow anonymous choices
+codegen-c’s DFDL subset doesn’t allow anonymous choices
and DFDL schema authors should write their schema like this:</p>
</div>
<div class="listingblock">
@@ -236,73 +236,32 @@ every use of _choice very carefully.</p>
<div class="sect2">
<h3 id="arrays">Arrays</h3>
<div class="paragraph">
-<p>Currently we create an ERD for an array with the array’s name
-and the scalar type of its first element,
-but the ERD has no numChildren and the rest of its fields are NULL.
-Then in the parent element’s ERD, we expand and inline the array
-into the parent element’s offsets and childrenERDs
-with incrementing offsets for each array element
-and the same pointer to the same array ERD for each array element.
-We also expand and inline the array
-into the parent element’s parseSelf and unparseSelf functions
-with as many parse and unparse calls as there are array elements.</p>
+<p>We inline an array’s minOccurs and maxOccurs
+into the array’s parseSelf and unparseSelf functions
+and throw an error if the count isn’t within these bounds.
+The reason why we perform these checks is because
+we inline arrays' maximum size into C structs
+and we put C structs into static memory to avoid using
+heap memory which may not be available on all platforms.</p>
</div>
<div class="paragraph">
-<p>We need to change this approach to handle arrays
-having undetermined lengths at compile time.
-One possible approach might be to define an ERD for an array
-like an ERD for a complex element with one child.
-The typeCode might become ARRAY or remain COMPLEX,
-the numChildren would be 1,
-the offsets would be the offset of the first array element
-(allowing room to skip over an actual number of elements
-stored in the C struct to the offset of the actual array,
-or to point to memory allocated from the heap),
-the childrenERDs would be the ERD of the first array element,
-the parseSelf would be a function to parse all array members,
-and the unparseSelf would be a function to unparse all array members.
-These functions would know how to find the number of elements
-depending on dfdl:occursCountKind when parsing
-(fixed, implicit, parsed, expression, or stopValue)
-and depending on a count stored in the C struct when unparsing.
-These functions also would know how to loop as many times
-as needed to parse or unparse each array element using the
-first array element’s ERD in childrenERDs every time.</p>
-</div>
-<div class="paragraph">
-<p>Note that we don’t have to store a count
-of the actual number of array elements in the C struct
-for a dfdl:occursCountKind of fixed, expression, or stopValue.
-Fixed means the count is a known constant at compile time.
-Expression means the count is already stored in
-another C struct field which we just have to find
-via the expression when parsing and unparsing.
-StopValue means we only need to look inside the array
-for a stopValue when parsing and unparsing.
-However, we do need to store an actual count in the C struct
-for a dfdl:occursCountKind of implicit or parsed
-because we will have no other possible way
-to find the actual count when unparsing.
-Our C code also should allow the count to be zero
-without the code blowing up.</p>
-</div>
-<div class="paragraph">
-<p>If we want the C code to validate the array’s count
-against the array’s minOccurs and maxOccurs,
-we can inline the array’s minOccurs and maxOccurs
-into the array’s parseSelf and unparseSelf functions.
-However, we should allow the normal case to be no validation,
-since Daffodil must not enforce min/maxOccurs
-if the user wants to parse and unparse well-formed but invalid data
-for forensic analysis.
-However, we still can let min/maxOccurs influence the generated C code.
+<p>In the normal case, Daffodil’s Scala backend parses and unparses
+unbounded or well-formed arrays without enforcing min/maxOccurs
+for forensic analysis and easier debugging.
+If we want the C backend to parse and unparse unbounded arrays
+and well-formed arrays, we still can inline min/maxOccurs into
+the generated C code without enforcing their bounds.
If maxOccurs is unbounded or the largest possible array size
-(maxOccurs - minOccurs) is larger than a heuristic or tunable,
-we should allocate storage for the array from the heap
-instead of declaring storage for the array inline in the C struct.
-The normal case should be to inline the array into the C struct
-with the array’s maximum size since bare metal C and VHDL
-will not be able to allocate memory from a heap dynamically.</p>
+(maxOccurs - minOccurs) is larger than a tunable,
+we would allocate storage for the array from the heap
+instead of inlining the array’s storage into the C struct.
+If the array is small enough to inline into the C struct
+but the array needs more space than its inlined space,
+we can switch that array to heap memory at runtime as long as
+we track its heap/inline status in another C struct field.
+We still should keep inlining finite bounded arrays into C structs
+since some embedded systems will not be able to allocate memory
+from a heap dynamically.</p>
</div>
</div>
<div class="sect2">
@@ -403,7 +362,7 @@ like an offline log processor does.</p>
<div class="sect2">
<h3 id="recovering-after-errors">Recovering after errors</h3>
<div class="paragraph">
-<p>As we continue to build out runtime2,
+<p>As we continue to build out codegen-c,
we may need to distinguish more types of errors
and allow backtracking and retrying.
Right now we handle only parse/unparse and
@@ -455,7 +414,7 @@ but we would need a good use case requiring recovering after errors.</p>
<h3 id="validate-fixed-values-in-runtime1-too">Validate "fixed" values in runtime1 too</h3>
<div class="paragraph">
<p>If we change runtime1 to validate "fixed" values
-like runtime2 does, then we can resolve
+like codegen-c does, then we can resolve
<a href="https://issues.apache.org/jira/browse/DAFFODIL-117">DAFFODIL-117</a>.</p>
</div>
</div>
@@ -480,7 +439,7 @@ matching the unparsed choice branch,
the writer must write an explicit dfdl:outputValueCalc
expression to replace the choiceDispatchKey
even though supporting dfdl:outputValueCalc
-in runtime2 is likely a distant goal.</p>
+in codegen-c is likely a distant goal.</p>
</div>
</div>
</div>
diff --git a/content/releases/3.1.0/index.html b/content/releases/3.1.0/index.html
index 5e08ad7..3355d7d 100644
--- a/content/releases/3.1.0/index.html
+++ b/content/releases/3.1.0/index.html
@@ -254,12 +254,13 @@ Run the folllowing command:
<h3 id="release-notes">Release Notes</h3>
-<h4 id="code-generation-backend-runtime2">Code Generation Backend (runtime2)</h4>
+<h4 id="code-generation-backend">Code Generation Backend</h4>
-<p>A new code generation back-end, called <code class="language-plaintext highlighter-rouge">runtime2</code>, is added supporting generating
-C code with limited parse and unparse capabilities. See the <a href="/cli">Command Line
-Interface</a> page for commands and options to generate code. Visit
-<a href="/dev/design-notes/runtime2-todos/">Runtime2 ToDos</a> for a list of limitations
+<p>A new code generation backend, called <code class="language-plaintext highlighter-rouge">daffodil-runtime2</code>, is added
+supporting generating C code with limited parse and unparse
+capabilities. See the <a href="/cli">Command Line Interface</a> page for commands
+and options to generate code. Visit <a href="/dev/design-notes/daffodilc-todos/">DaffodilC
+ToDos</a> for a list of limitations
and planned future work.</p>
<ul>
@@ -423,7 +424,7 @@ least 1 byte of data is available to parse, or end-of-file is reached.</p>
<li>Scallop 4.0.2 <small>(update)</small></li>
</ul>
-<p><strong>Code Generator (runtime2)</strong></p>
+<p><strong>Code Generator</strong></p>
<ul>
<li>OS-Lib 0.7.6 <small>(new)</small></li>
diff --git a/content/releases/3.2.0/index.html b/content/releases/3.2.0/index.html
index 1560cbb..52aa674 100644
--- a/content/releases/3.2.0/index.html
+++ b/content/releases/3.2.0/index.html
@@ -319,9 +319,9 @@ InfosetOutputter classes.</p>
<li>[<a href="https://issues.apache.org/jira/browse/DAFFODIL-2558">DAFFODIL-2558</a>] Update GitHub Actions Workflow to use Java 17</li>
</ul>
-<h4 id="code-generation-backend-runtime2">Code Generation Backend (runtime2)</h4>
+<h4 id="code-generation-backend">Code Generation Backend</h4>
-<p>The C code generation back-end, called <code class="language-plaintext highlighter-rouge">runtime2</code> has had these enhancements.</p>
+<p>The C code generation backend called <code class="language-plaintext highlighter-rouge">daffodil-runtime2</code> has had these enhancements.</p>
<ul>
<li>[<a href="https://issues.apache.org/jira/browse/DAFFODIL-2566">DAFFODIL-2566</a>] Extend C code generator to support hexBinary elements</li>
@@ -445,7 +445,7 @@ vulerabilities.</p>
<li>Scallop 4.1.0 <small>(update)</small></li>
</ul>
-<p><strong>Code Generator (runtime2)</strong></p>
+<p><strong>Code Generator</strong></p>
<ul>
<li>OS-Lib 0.7.8 <small>(update)</small></li>
diff --git a/content/releases/3.2.1/index.html b/content/releases/3.2.1/index.html
index 08ce16e..94ee1f3 100644
--- a/content/releases/3.2.1/index.html
+++ b/content/releases/3.2.1/index.html
@@ -306,7 +306,7 @@ This has been fixed.</p>
<li>JDOM2 2.0.6.1 <small>(update)</small></li>
</ul>
-<p><strong>Code Generator (runtime2)</strong></p>
+<p><strong>Code Generator</strong></p>
<ul>
<li>OS-Lib 0.8.0 <small>(update)</small></li>
diff --git a/content/releases/3.3.0/index.html b/content/releases/3.3.0/index.html
index 7d625f1..0aab5c7 100644
--- a/content/releases/3.3.0/index.html
+++ b/content/releases/3.3.0/index.html
@@ -321,7 +321,7 @@ functionality of the prior release.</p>
<li>Xerces 2.12.2 <small>(update)</small></li>
</ul>
-<p><strong>Code Generator (runtime2)</strong></p>
+<p><strong>Code Generator</strong></p>
<ul>
<li>OS-Lib 0.8.1 <small>(update)</small></li>
diff --git a/content/releases/3.4.0/index.html b/content/releases/3.4.0/index.html
index 046d8b3..956e05e 100644
--- a/content/releases/3.4.0/index.html
+++ b/content/releases/3.4.0/index.html
@@ -303,7 +303,7 @@ original data.</p>
<li>[<a href="https://issues.apache.org/jira/browse/DAFFODIL-2708">DAFFODIL-2708</a>] XML String feature in XML Text Infoset Inputter/Outputter</li>
</ul>
-<h4 id="c-code-generator-runtime2-updates">C Code Generator (Runtime2) Updates</h4>
+<h4 id="c-code-generator-updates">C Code Generator Updates</h4>
<p>The C code generator backend now supports reading and writing N-bit booleans
and integers, where N is an explicit length from 1 to 64 bits. Additional