[GitHub] [arrow] zhztheplayer edited a comment on pull request #7030: ARROW-7808: [Java][Dataset] Implement Datasets Java API by JNI to C++

[GitBox <gi...@apache.org>]

zhztheplayer edited a comment on pull request #7030:
URL: https://github.com/apache/arrow/pull/7030#issuecomment-644494295


   @fsaintjacques Rebased now. 
   
   As we have a unresolved [discussion](https://github.com/apache/arrow/pull/7030#discussion_r440075959) about memory management of native buffers. Should we clarify a bit more before writing any fixes? (also ping @jacques-n , @emkornfield)
   
   The question is what's the major problem now? I list some that I can see in comments:
   
   1. We are package-hacking `org.apache.arrow.memory` in module `arrow-dataset`.
   Yes `Ownerships.java` uses some hacks, so I've removed the class in latest commits (the whole class is not necessary for current use case). Now only `NativeUnderlyingMemory.java` is still in the package `org.apache.arrow.memory`. 
   2. We lose java native heap caps.
   Can we clarify this? I think the explanation can be either we lose caps from JVM direct memory, or we lose caps from BufferAllocator. For the latter one I think buffers are already registered to allocator.
   3. OOM-check is proceed after buffer is created.
   Yes. But is this really a big problem? see [comment](https://github.com/apache/arrow/pull/7030#discussion_r440075959).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org