[GitHub] [arrow-rs] jorgecarleitao commented on issue #20: FFI listarray lead to UB.

[GitBox <gi...@apache.org>]

jorgecarleitao commented on issue #20:
URL: https://github.com/apache/arrow-rs/issues/20#issuecomment-831365431


   I have spent my weekend on this problem 😭
   
   I now understand what we are doing wrong, but the solution requires a major design change of the FFI.
   
   This gist is that we currently treat children as independent from the parent and `drop` them when no longer in use, which is out of spec.
   
   The C data interface requires (MUST) that the parent is responsible for all child deallocations. When we destruct a list array imported from C++, the following currently happens:
   
   1. the list array `drop` is called
   2. the child array `drop` is called
   3. the child array FFI `release` is called
   4. the list array FFI `release` is called, calling the child array FFI `release` (as per spec), causing a double free
   
   We should remove step 3 from this sequence, and let step 4 do its job.
   
   However, for this, we need to figure out a way to share a ref counted owned parent to all child arrays, so that when the last array is deallocated, we can release the parent and all associated child arrays on the same `release`. Together with this, we must consider all imported child ffi arrays as non-owned (e.g. refs), so that they do not call their own `release` on `drop`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org