You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by sl...@apache.org on 2019/02/14 17:43:19 UTC
[incubator-daffodil-site] 05/39: Misc updates to website
This is an automated email from the ASF dual-hosted git repository.
slawrence pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-daffodil-site.git
commit 4327a3a08170435719745b4349bda4455be9e2f0
Author: Steve Lawrence <sl...@tresys.com>
AuthorDate: Thu Jan 18 13:15:32 2018 -0500
Misc updates to website
- Change how external links are shown. Now just need to add the external
class to <a> tags instead of inserting an empty <i> tag.
- Add page describing Daffodil extensions to DFDL
- Add examples page
- Modify Getting Started to include NiFi and Calabash and reword some
things to be more clear
---
site/_includes/themes/apache/_navigation.html | 17 +--
site/assets/themes/apache/css/style.css | 4 +-
site/dfdl-extensions.md | 57 ++++++++++
site/examples.md | 158 ++++++++++++++++++++++++++
site/getting-started.md | 30 ++++-
5 files changed, 251 insertions(+), 15 deletions(-)
diff --git a/site/_includes/themes/apache/_navigation.html b/site/_includes/themes/apache/_navigation.html
index 389183a..9232c9a 100644
--- a/site/_includes/themes/apache/_navigation.html
+++ b/site/_includes/themes/apache/_navigation.html
@@ -19,6 +19,7 @@
<a href="#" data-toggle="dropdown" class="dropdown-toggle">Docs<b class="caret"></b></a>
<ul class="dropdown-menu dropdown-left">
<li><a href="/getting-started/">Getting Started</a></li>
+ <li><a href="/examples/">Examples</a></li>
<li><a href="/docs/latest/javadoc/">Java API</a></li>
<li><a href="/docs/latest/scaladoc/">Scala API</a></li>
<li><a href="/docs/dfdl/">DFDL Specification</a></li>
@@ -36,19 +37,19 @@
<li id="development">
<a href="#" data-toggle="dropdown" class="dropdown-toggle">Development<b class="caret"></b></a>
<ul class="dropdown-menu dropdown-left">
- <li><a href="https://cwiki.apache.org/confluence/display/DAFFODIL/Code+Contributor+Workflow"><i class="external"></i>Contributor Workflow</a></li>
- <li><a href="https://github.com/apache/incubator-daffodil"><i class="external"></i>GitHub</a></li>
- <li><a href="https://issues.apache.org/jira/projects/DAFFODIL/"><i class="external"></i>JIRA</a></li>
+ <li><a class="external" href="https://cwiki.apache.org/confluence/display/DAFFODIL/Code+Contributor+Workflow">Contributor Workflow</a></li>
+ <li><a class="external" href="https://github.com/apache/incubator-daffodil">GitHub</a></li>
+ <li><a class="external" href="https://issues.apache.org/jira/projects/DAFFODIL/">JIRA</a></li>
</ul>
</li>
<li id="apache">
<a href="#" data-toggle="dropdown" class="dropdown-toggle">Apache<b class="caret"></b></a>
<ul class="dropdown-menu">
- <li><a href="http://www.apache.org/"><i class="external"></i>Apache Software Foundation</a></li>
- <li><a href="http://www.apache.org/licenses/"><i class="external"></i>License</a></li>
- <li><a href="http://www.apache.org/security"><i class="external"></i>Security</a></li>
- <li><a href="http://www.apache.org/foundation/sponsorship.html"><i class="external"></i>Sponsorship</a></li>
- <li><a href="http://www.apache.org/foundation/thanks.html"><i class="external"></i>Thanks</a></li>
+ <li><a class="external" href="http://www.apache.org/">Apache Software Foundation</a></li>
+ <li><a class="external" href="http://www.apache.org/licenses/">License</a></li>
+ <li><a class="external" href="http://www.apache.org/security">Security</a></li>
+ <li><a class="external" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a class="external" href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
</ul>
</li>
</ul>
diff --git a/site/assets/themes/apache/css/style.css b/site/assets/themes/apache/css/style.css
index 719febf..74849fd 100644
--- a/site/assets/themes/apache/css/style.css
+++ b/site/assets/themes/apache/css/style.css
@@ -403,7 +403,7 @@ hr {
}
}
-i.external {
+a.external:before {
position: relative;
top: 1px;
display: inline-block;
@@ -416,7 +416,7 @@ i.external {
margin-right: 5px;
}
-i.external:before {
+a.external:before {
content: "\e066";
}
diff --git a/site/dfdl-extensions.md b/site/dfdl-extensions.md
new file mode 100644
index 0000000..665df5d
--- /dev/null
+++ b/site/dfdl-extensions.md
@@ -0,0 +1,57 @@
+---
+layout: page
+title: DFDL Extensions
+group: nav-right
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+## DFDL Extensions
+
+Daffodil provides extensions to the DFDL specification. These properties are in the namespace defined by the URI ``urn:ogf:dfdl:2013:imp:daffodil.apache.org:2018:ext``. To use these the Daffodil DFDL Extensions, bind this namespace to the ``daf`` prefix in the ``xs:schema`` element, like so:
+
+``` xml
+<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
+ xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/"
+ xmlns:daf="urn:ogf:dfdl:2013:imp:daffodil.apache.org:2018:ext">
+```
+
+The following symbols defined in this namespace are described below.
+
+### Expression Functions
+
+``daf:error()``
+
+ : A function that can be used in DFDL expressions. This functions does not return a value or accept any arguments. When called, it causes a Parse Error or Unparse Error.
+
+ *This function is deprecated as of Daffodil 2.0.0. Use the ``fn:error(...)`` function instead.*
+
+``daf:trace($value, $label)``
+
+ : A function that can be used in DFDL expressions, similar to the ``fn:trace()`` function. This logs the string ``$label`` followed by ``$value`` converted to a string and returns ``$value``. The second argument must be of type ``xs:string``.
+
+### Properties
+
+``daf:parseUnparsePolicy``
+
+ : A property applied to simple and complex elements, which specifies whether the element supports only parsing, only unparsing, or both parsing and unparse. Valid values for this property are ``parse``, ``unparse``, or ``both``. This allows one to leave off properties that are required for only parse or only unparse, such as ``dfdl:outputValueCalc`` or ``dfdl:outputNewLine``, so that one may have a valid schema if only a subset of functionality is needed.
+
+ All elements must have a compatible parseUnparsePolicy with the compilation parseUnparsePolicy (which is defined by the root element daf:parseUnparsePolicy and/or the Daffodil parseUnparsePolicy tunable) or it is a Schema Definition Error. An element is defined to have a compatible parseUnparsePolicy if it has the same value as the compilation parseUnparsePolicy or if it has the value ``both``.
+
+ For compatibility, if this property is not defined, it is assumed to be ``both``.
diff --git a/site/examples.md b/site/examples.md
new file mode 100644
index 0000000..694fd56
--- /dev/null
+++ b/site/examples.md
@@ -0,0 +1,158 @@
+---
+layout: page
+title: Examples
+group: nav-right
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+
+## Examples
+
+Below are two examples of how Daffodil parses Comma-Separated Values (CSV) and Packet Capture data (PCAP) data to XML. For simplicity, each example uses the Daffodil [Command Line Interface](/cli) to parse input data to output XML.
+
+### Comma-Separated Values (CSV)
+
+This DFDL schema is found at the DFDLSchemas GitHub [CSV repository](https://github.com/DFDLSchemas/CSV).
+
+```bash
+$ daffodil parse --schema CSV/src/main/resources/com/tresys/csv/xsd/csv.dfdl.xsd CSV/src/test/resources/com/tresys/csv/data/simpleCSV.csv
+```
+The above command uses the csv.dfdl.xsd schema to parse the simpleCSV.csv input data. The simpleCSV.csv contains the following data:
+
+```
+last,first,middle,DOB
+smith,robert,brandon,1988-03-24
+johnson,john,henry,1986-01-23
+jones,arya,cat,1986-02-19
+```
+
+The result of the parse is the following XML:
+
+```xml
+<ex:file xmlns:ex="http://example.com">
+ <header>
+ <title>last</title>
+ <title>first</title>
+ <title>middle</title>
+ <title>DOB</title>
+ </header>
+ <record>
+ <item>smith</item>
+ <item>robert</item>
+ <item>brandon</item>
+ <item>1988-03-24</item>
+ </record>
+ <record>
+ <item>johnson</item>
+ <item>john</item>
+ <item>henry</item>
+ <item>1986-01-23</item>
+ </record>
+ <record>
+ <item>jones</item>
+ <item>arya</item>
+ <item>cat</item>
+ <item>1986-02-19</item>
+ </record>
+</ex:file>
+```
+
+### Packet Capture Data (PCAP)
+
+This DFDL schema is found in the DFDLSchemas GitHub [PCAP repository](https://github.com/DFDLSchemas/PCAP).
+
+The PCAP file format is a binary file format used to capture network packets. For information on this file format, visit the [Wireshark Libpcap File Format](http://wiki.wireshark.org/Development/LibpcapFileFormat) page.
+
+```bash
+$ daffodil parse --schema PCAP/src/main/resources/com/tresys/pcap/xsd/pcap.dfdl.xsd PCAP/src/test/resources/com/tresys/pcap/data/icmp.cap
+```
+
+The above command uses the pcap.dfdl.xsd schema to parse the icmp.cap input file. Viewed as a hex dump, the icmp.cap file looks like:
+
+```
+0000000 c3d4 a1b2 0002 0004 0000 0000 0000 0000
+0000020 ffff 0000 0001 0000 6fc4 51c1 ccf8 000c
+0000040 004a 0000 004a 0000 5000 e056 4914 0c00
+...
+0001300 0000 5c2f 0002 0024 6261 6463 6665 6867
+0001320 6a69 6c6b 6e6d 706f 7271 7473 7675 6177
+0001340 6362 6564 6766 6968
+```
+
+The result of the parse is the following XML:
+
+```xml
+<pcap:PCAP xmlns:pcap="urn:pcap:2.4">
+ <PCAPHeader>
+ <MagicNumber>D4C3B2A1</MagicNumber>
+ <Version>
+ <Major>2</Major>
+ <Minor>4</Minor>
+ </Version>
+ <Zone>0</Zone>
+ <SigFigs>0</SigFigs>
+ <SnapLen>65535</SnapLen>
+ <Network>1</Network>
+ </PCAPHeader>
+ <Packet>
+ <PacketHeader>
+ <Seconds>1371631556</Seconds>
+ <USeconds>838904</USeconds>
+ <InclLen>74</InclLen>
+ <OrigLen>74</OrigLen>
+ </PacketHeader>
+ <pcap:LinkLayer>
+ <pcap:Ethernet>
+ <MACDest>005056E01449</MACDest>
+ <MACSrc>000C29340BDE</MACSrc>
+ <Ethertype>2048</Ethertype>
+ <pcap:NetworkLayer>
+ <pcap:IPv4>
+ <IPv4Header>
+ <Version>4</Version>
+ <IHL>5</IHL>
+ <DSCP>0</DSCP>
+ <ECN>0</ECN>
+ <Length>60</Length>
+ <Identification>55107</Identification>
+ <Flags>0</Flags>
+ <FragmentOffset>0</FragmentOffset>
+ <TTL>128</TTL>
+ <Protocol>1</Protocol>
+ <Checksum>11123</Checksum>
+ <IPSrc>192.168.158.139</IPSrc>
+ <IPDest>174.137.42.77</IPDest>
+ </IPv4Header>
+ <PayloadLength>40</PayloadLength>
+ <Protocol>1</Protocol>
+ <pcap:ICMPv4>
+ <Type>8</Type>
+ <Code>0</Code>
+ <Checksum>10844</Checksum>
+ <Data>02002100</Data>
+ </pcap:ICMPv4>
+ </pcap:IPv4>
+ </pcap:NetworkLayer>
+ </pcap:Ethernet>
+ </pcap:LinkLayer>
+ </Packet>
+ ...
+</pcap:PCAP>
+```
diff --git a/site/getting-started.md b/site/getting-started.md
index 8f619fe..e46a060 100644
--- a/site/getting-started.md
+++ b/site/getting-started.md
@@ -26,22 +26,42 @@ limitations under the License.
### Using Daffodil
-Daffodil has two methods in which it can be invoked:
+Daffodil is a library, requiring Java 8, used to convert between fixed format data and XML/JSON based on a DFDL schema. Some [examples](/examples) show the result of Daffodil parsing various inputs into XML. Multiple mechanisms that use the Daffodil library invoke its capabilities are described below.
-* Via the [Command Line Interface](/cli). Capabilities include parsing, unaparsing, an interactive debugger, and more.
-* Programmatically via the [Java API](/docs/latest/javadoc) or [Scala API](/docs/latest/scaladoc). Examples for using the Java API are available on the [OpenDFDL examples](https://github.com/OpenDFDL/examples.git) repository (not Apache affiliated). The Daffodil [releases](/releases) describe how to include a dependency to Daffodil via maven and SBT.
+[Command Line Interface](/cli)
-Both methods require Java 8.
+ : Linux and Windows command line tool with capabilities include parsing, unaparsing, an interactive debugger, and more. Available via download in binary [releases](/releases).
+
+[Java API](/docs/latest/javadoc) or [Scala API](/docs/latest/scaladoc)
+
+ : Examples for using the Java API are available on the [OpenDFDL examples](https://github.com/OpenDFDL/examples.git) repository. Individual [releases](/releases) describe how to include a dependency to Daffodil via Maven and SBT.
+
+[Apache NiFi Processors](https://opensource.ncsa.illinois.edu/bitbucket/projects/DFDL/repos/daffodil-nifi/browse)
+
+ : Provides two [Apache NiFi](https://nifi.apache.org/) processors for parsing and unparsing NiFi FlowFiles.
+
+[XML Calabash Extension](https://opensource.ncsa.illinois.edu/bitbucket/projects/DFDL/repos/daffodil-nifi/browse)
+
+ : An extension to [XML Calabash](http://xmlcalabash.com) that declares XProc pipeline steps to parse and unparse input data.
### Existing DFDL Schemas
-Multiple DFDL schemas exist on the [DFDLSchemas github](https://github.com/DFDLSchemas) (not Apache affiliated) that are available to download and try out. This includes multiple image formats, financial transaction formats, and more.
+Many DFDL schemas are freely and publicly available at [DFDLSchemas](https://github.com/DFDLSchemas) on GitHub. This includes image formats, financial transaction formats, healthcare formats, and more. Some DFDL Schemas for Controlled Unclassified Information (CUI) data formats are available on [DI2E.net](https://confluence.di2e.net/pages/viewpage.action?pageId=196139975), which requires US Citizenship and DoD sponsorship.
+
### Creating DFDL Schemas
+#### Layout
+
When creating a custom DFDL schema, it is beneficial to follow the directory/file layout described in [Standard DFDL Schema Project Layout](/dfdl-layout). This layout provides a consistent and familiar structure for DFDL schema development that existing tooling understands, provides for easy testing and schema packaging, and ensures no name conflicts on classpaths if multiple schemas are used together.
+#### Editors
+
Any text editor can be used for authoring DFDL schemas. But because DFDL schemas are a subset of XML Schema, using an XML editor (especially an XML Schema aware editor) can provide intelligent assistance in authoring DFDL Schemas, including attribute and tag completion, validation, indentation, visualizations, etc. Below are resources for configuring various tools specifically for editing or authoring DFDL schemas.
* [Eclipse IDE](/eclipse-configuration)
+#### DFDL Extensions
+
+Daffodil provides extensions to the DFDL specification to add extra enhancements and capabilities. These extensions are listed at [DFDL Extensions](/dfdl-extensions).
+