[GitHub] [arrow] zhztheplayer commented on pull request #10201: ARROW-11776: [Java][Dataset] Support writing to files within dataset scanner via JNI

[GitBox <gi...@apache.org>]

zhztheplayer commented on pull request #10201:
URL: https://github.com/apache/arrow/pull/10201#issuecomment-846405682


   @emkornfield 
   
   > Refactoring to allow passing native java buffers to C++ code. For this, another option that might be simpler/more reliable is to try to use the Arrow C ABI to translate the data across java and C++. Did you consider this?
   
   Once we had a short discussion around this https://issues.apache.org/jira/browse/ARROW-7272?focusedCommentId=16983849&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16983849
   
   C data interface does sound to be better. Although I believe a Java implementation will require a design that probably tightly relies on JNI which makes the implementation slightly more complex than in other language's case. I think we can open a separate JIRA ticket for that to do the work in future as we don't yet have the implementation in Java. So far I have extracted the [commit](https://github.com/apache/arrow/pull/10201/commits/03e3c2c924f1f4f21a66b5c4cec1d4ab2d3fee18) that refactored current code within flatbuffers and marked it to resolve the issue ARROW-7272 (if it's OK I may open an individual PR within the commit for 7272). What do you think?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org