[GitHub] stevedlawrence opened a new pull request #82: Support the --stream option with the unparse subcommand

[GitBox <gi...@apache.org>]

stevedlawrence opened a new pull request #82: Support the --stream option with the unparse subcommand
URL: https://github.com/apache/incubator-daffodil/pull/82
 
 
   - When the --stream option is provided to the parse command, output a
     NUL character (\u0000) after each infoset is output to act as a
     separator. The NUL character is not allowed in either XML or JSON
     output, so this can never be confused with infoset data. This is also
     consistent with common linux tools like 'find' and 'xargs' which can
     use the NUL character for data separation.
   - When the --stream option is provided to the unparse command, the
     input data is split on the NUL character. Each chunk is converted to
     an InfosetInputter and unparsed to the same output channel. Note that
     we need to do this split because all the XML parsing libraries we use
     see the XML stream as invalid since there is not a single root
     element.
   - Fix a bug in the DirectOrBufferDataOutputStream where we incorrectly
     closed the underlying JavaOutputStream provided from the user, which
     caused successive unparse writes to fail. We do still need to close
     JavaOutputStreams related to layering, so add a flag that defines
     which DOS's are for layering and close those when appropriate. But if
     a DOS is not for layering, do not close it--the user is responsible
     for cleanup.
   
   These changes enable piping the output of parse --stream to unparse
   --stream, acting as a streaming round trip.
   
   DAFFODIL-1967

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services