[GitHub] [incubator-daffodil] stevedlawrence opened a new pull request #280: Remove the 2GB limit when unparsing BLOBs

[GitBox <gi...@apache.org>]

stevedlawrence opened a new pull request #280: Remove the 2GB limit when unparsing BLOBs
URL: https://github.com/apache/incubator-daffodil/pull/280
 
 
   Due to suspensions, it is possible that we could unparse a BLOB to a
   buffered DataOutputStream, which means the entire contents of the BLOB
   would be stored in memory until the suspension and its DataOutputStream
   were finished. Not only is this a waste of memory since the blob is all
   on disk, but can also limit BLOBs to 2GB since buffered
   DataOutputStreams use a Byte array for buffering.
   
   This resolves this issue by not immediately unparsing BLOB data, but
   instead adding a new BLOB DataOutputStream which writes BLOB data to the
   previous DataOutputStream once that stream becomes direct. This means we
   never store the full BLOB data in memory and remove the 2GB limit.
   
   DAFFODIL-2194

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services