[jira] [Created] (ARROW-15275) [C++][JNI] DisposableScannerAdaptor does not handle arrays with offsets

["Weston Pace (Jira)" <ji...@apache.org>]

Weston Pace created ARROW-15275:
-----------------------------------

             Summary: [C++][JNI] DisposableScannerAdaptor does not handle arrays with offsets
                 Key: ARROW-15275
                 URL: https://issues.apache.org/jira/browse/ARROW-15275
             Project: Apache Arrow
          Issue Type: Bug
          Components: C++
            Reporter: Weston Pace


The DisposableScannerAdaptor is a JNI bridge from Java to the C++ datasets API.  When it scans record batches it collects all of the buffers from all of the arrays and returns a list of buffer handles to Java which puts these into an ArrowRecordBatch on the Java end.

Unfortunately, if the array has offsets then the bridge does not return the offset buffer but it returns the entire buffer.  The Java record batch is then incorrect.  The length is wrong (and so it doesn't fully free the memory) and the values are incorrect.

I'm not familiar enough with the Java implementation to suggest a good fix.  Figuring out the buffer offsets from array offsets is a bit tricky since the logic depends on the data type.  Also, I'm pretty sure the Java side now has to take ownership of the entire buffer which could be tricky because multiple batches could share ownership of the buffer.

As a temporary fix for ARROW-13554 I am going to copy the array if it has an offset.  This means the transfer is not zero-copy so I'm creating this issue to solve this properly.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)