Streaming Parsing Example

["Beckerle, Mike" <mb...@owlcyberdefense.com>]

I have seen enough interest in how to do streaming parsing of data properly using Daffodil that I created a self-contained example of this in Java.

The example is written in Java, called hexWords, and is available at

https://github.com/OpenDFDL/examples/tree/master/hexWords

There are two classes in it of interest.

MessageParser is a class that one creates using a DFDL schema, which it compiles, and then you supply an input stream, and call parse() repeatedly to pull messages one-by-one from the input stream.

The HexWordsStreamingApp class is the simulated application that uses the MessageParser class.

The DFDL schema for the toy-but-interesting hexWords format is provided.

Of interest is the way this schema converts structural problems with the data into creation of <illegal> elements, which capture the faulty data, but are always invalid given XML Schema validation. This technique allows recovery from many kinds of parse errors, which enables parsing to continue and hopefully get a subsequent successful message parse that creates a valid parse result.

Mike Beckerle | Principal Engineer

[cid:6bd77c27-b4a6-476a-b53c-bfcab38e4557]

mbeckerle@owlcyberdefense.com<ma...@owlcyberdefense.com>

P +1-781-330-0412

Connect with us!

[cid:3cf5c304-2e30-41a7-a7b0-39dd0583bbf4]<https://www.linkedin.com/company/owlcyberdefense/>[cid:848655af-1bcc-415f-959c-3936532abf9f]<https://twitter.com/owlcyberdefense>

[cid:34c3471f-4406-4059-9e11-d426fa8f9bb1]<https://owlcyberdefense.com/resources/events/>



The information contained in this transmission is for the personal and confidential use of the individual or entity to which it is addressed. If the reader is not the intended recipient, you are hereby notified that any review, dissemination, or copying of this communication is strictly prohibited. If you have received this transmission in error, please notify the sender immediately