You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@arrow.apache.org by "Paul Taylor (JIRA)" <ji...@apache.org> on 2019/04/03 06:59:00 UTC

[jira] [Created] (ARROW-5100) [JS] Writer swaps byte order if buffers share the same underlying ArrayBuffer

Paul Taylor created ARROW-5100:
----------------------------------

             Summary: [JS] Writer swaps byte order if buffers share the same underlying ArrayBuffer
                 Key: ARROW-5100
                 URL: https://issues.apache.org/jira/browse/ARROW-5100
             Project: Apache Arrow
          Issue Type: Bug
          Components: JavaScript
    Affects Versions: 0.13.0
            Reporter: Paul Taylor
            Assignee: Paul Taylor
             Fix For: 0.14.0


We collapse contiguous Uint8Arrays that share the same underlying ArrayBuffer and have overlapping byte ranges. This was done to maintain true zero-copy behavior when using certain node core streams that use a buffer pool internally, and could write chunks of the same logical Arrow Message at out-of-order byte offsets in the pool.

Unfortunately this can also lead to a bug where, in rare cases, buffers are swapped while writing Arrow Messages too. We could have a flag to indicate whether we think collapsing out-of-order same-buffer chunks is safe, but I'm not sure if we can always know that, so I'd prefer to take it out and incur the copy cost.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)